XTEND functioning

The dictionary

The dictionary contains the definition of all the XTEND setup records.

It is formed of a set of XML files (on by XTEND function) which are stored on the server of the X3 solution in the directory:
X3SOL/dossiers/X3_PUB/X3FOLDER/X_TEND/X_GEN/SITE.

Validation

When the developer validates a record, a processing generating the dictionary is activated to create the xml file.

The validation generates a new XML dictionary of XTEND setups.

The new dictionary is only automatically taken into account on reload/F5 of the HTML page in the browser if the option of the setup record of the site, function Websites (GESAYS)'Technique\Check updates\Web dictionary' is checked.

Otherwise, the reloading of the dictionary needs to be forced using the following url:
'http://hostname:port/xtend/svc/X3Solution/X3Folder/XtendSite/admin/reposit/reload'

After modification of X3 setups:

It is advised to validate the site as a whole in order to build the XTEND server dictionary via function Website validation (AYTFCYGEN).

Verify that the XTEND site is published, that is that the field "Publish the site" of the "Web site" record is checked.

Access to X3 files

The XTEND server accesses the X3 files via the HHTP server (Apache) which is installed by default on the workstation hosting the main X3 server.

The root directory is the X3_PUB directory of the solution.

The update detection is performed upon reading the 'TimeStamp' (date of last modification) the file.

The XTEND server reads XML files and builds a database in memory to optimize the setup access.

Upon loading, XTEND controls the consistency of the dictionary and generates an exception in the case of errors.

In the case of a dictionary read error

Verify that the unregistered users can access the X3_PUB directory
The url address http://hostX3/Adonix_X3SOL/X3FOLDER/X_TEND/X_GEN/XTENDSITE/ must return the list of xml files
Under UNIX verify the user access rights

Optimization

By default, the dictionary and the local menus are stored on the X3 server.

The console setups xtend.server.reposit.local=off and xtend.server.menux3.local=off are used to define a local location for the files on the X3WEB server (optimization).

In this case, it is necessary to copy the dictionary and/or local menu directories under \WebData\LOCAL\X3SOLUTION\X3_PUB\.

WebData represents the 'Data' directory defined upon installation of the X3WEB server.

HTML processing

Read

By default, the pages and all the resources of the HTML project are stored on the X3 server in the directory:
/X3SOL/dossiers/X3_PUB/X3FOLDER/X_TEND/X_HTML/ASAMPLE/LANG/ (LANG being the language code of the site).

The developer must think about uploading the HTML pages to the X3 server before testing them in their browser.

Under unix it is necessary to verify that all the files of the HTML directory have the minimum rights upon 'read' for all users.

The process for the page reading and modification detection is the same as for the dictionary.

A setup of the XTEND site makes it possible to activate/deactivate the update verifications.

If the verification is deactivated, it is necessary to use the url to reload the dictionary to force the XTEND server to take into account the page modifications.

Parsing

A Web page is a text file containing HTML tags in which the developer added XTEND tokens.

After loading the page, the XTEND server reads the content and interprets the HTML and the tokens (parsing) to build a representation in memory of the page in the form of an object tree structure (DOM).

Each object represents a token and applies an HTML calculation method (computeHtml).

The tree structure of the tokens can be seen in the XTEND log file:
++ corresponds to the level - CXtdHtmBuild* is the object class

+CXtdHtmBuildDom - XTEND
++CXtdHtmBuildNodeHtml - HTML
++CXtdHtmBuildTagHead - <HEAD
+++CXtdHtmBuildTagBase - <BASE
++CXtdHtmBuildAdxScriptLibrary - ALIBJS
++CXtdHtmBuildTagHeadEnd - </HEAD
++CXtdHtmBuildTagBody - <BODY
++CXtdHtmBuildTagForm - <FORM
++CXtdHtmBuildAdxTagAnchor - AHOME
++CXtdHtmBuildAdxTagAnchor - AABOUT
++CXtdHtmBuildAdxCND - ADISPUSERLOGGEDIN
+++CXtdHtmBuildAdxTagAnchor - AUSERACCOUNT
+++CXtdHtmBuildAdxTagAnchor - ADLKLOGOUT
++CXtdHtmBuildAdxCND - ADISPUSERLOGGEDIN
+++CXtdHtmBuildAdxTagAnchor - ALOGIN
...

The HTML parser of the XTEND server is compatible with the version HTML 4.01 (DTD HTML 4.01 Transitional).

The only constraint imposed by XTEND for the HTML design is to use only one HTML form wrapping the whole body of the HTML page.
<body><form> Body content </form></body>

Construction

In order to calculate dynamically the content of the HTML page, the XTEND server carries out following operations:

1. Initializes the blocks

For interfaces of type data access:

Creates the query (selection) according to the setups
Calls the Web service via the interface associated with the block entity
Creates the entity 'Data access' in memory

For interfaces of type Action:

Reads the entities already present in memory

2. Goes through the objects of the tree structure (DOM) in a recursive manner and calls the method computeHtml

If the token is a field:

The method calculates the value of the field

If the token is a dynamic link:

The method calculates the url (tag <a></a>) or the JavaScript (tag <imput type="button")

If the token is a block:

The method carries out an iteration on all entities and calls the calculation entity of child tokens

If the token is a conditioned block:

The method evaluates the Boolean expression (criteria) and calls or not the calculation method for child tokens

3. The global result is the sum of the HTML produced by all the tokens.

The HTML calculation method depends on the type of token and HTML tag in which it is inserted.

HTML generation

The calculated HTML depends on the type of token and HTML tag in which it is inserted.

Field token in tags <td>,<div>, <span>, <p>...

Replaces the content of the tag by the value of the field

<td adx="myField"></td> ---> <td>Value</td>

Field token in a <select> tag

Selects the option corresponding to the value of the field by generating the selected attribute

Example if the value of myField is FR, and if the option value=FR and is selected (selected attribute)

<select adx="myField">
    <option value="AT">Austria</option>
    <option value="FR">France</option>
</select>

<select adx="myField">
    <option value="AT">Austria</option>
    <option selected value="FR">France</option>
</select>

Field token in a <img> tag

Generates the URL which makes it possible for the browser to read the picture

Dynamic link token in a <a> tag

If the token does not have an associated action (GET method) the href attribute of the tag is calculated
Otherwise (POST method), the tag is considered as a button (see below)

<a adx="MyDynLink">Logout</a>

<a href="http://host:port/xtend/page?DLK=MyDynLink&SOL=SOL...">Logout</a>

Dynamic link token in a <input type="button"> tag

Calculates the JavaScript xtdDoDlk function and inserts it in the 'onClick' attribute

The setups are contextual and managed by the XTEND library

Function xtdDoDlk

This function is generated by XTEND in the onClick attribute for dynamic links which have an associated action or selection.

It triggers the XTEND JavaScript process which processes user actions.

It is also used to find the data context (aCtxTag,aLineIdx) of the dynamic link, especially for tokens generated in the block lines.

function xtdDoDlk(aElmt,aDlk,aProtocol,aBlkId,aCtxTag,aLineIdx,aUrlAdd,aEvent,aSubmit,
aAutoId,aForceAcceptReload,aForceSelBlock,aCheckWebFields)
{/*
aElmt: JavaScript Object on which the user clicked
aDlk: Code of the clicked dynamic link
aProtocol: If !=null forces the protocol (https,http) for this link
aBlkId: Code of the reference block if adx='MyBlocRefence.MyLink'
aCtxTag: Identifier of the data context (if action with setups of type XTEND fields)
aLineIdx: Index of the line if the token is located in a block {{multi}}
aUrlAdd: Contains the value of the HTML setup xanchor adx="MyLink:xanchor=#section"
aEvent: Contains the JavaScript event object
aSubmit: True to submit the form
aAutoId: Id generated by XTEND if adx="MyLink:xautoid"
aForceAcceptReload:True to accept the reload (F5)
aForceSelBlock:Id of the block which processes the query if != AMAIN
aCheckWebFields:True to control the Web fields
*/
}

The data context of the page is stored in the page itself in a hidden span tag.
It contains all the field values (type 'Field token') used as action setup for the dynamic links of the page.
This method makes it possible to manage very easily the back option via the 'Back' function of the browser.

<span id="xtdctx" style="display: none; clear: both;">
XML containing the context of the page
</span>

Server data context

In order to fully understand how XTEND works it is necessary to detail the process defining the value of a field by the XTEND server.

The XTEND server keeps a data context which is managed in a stack of data blocks (entities).

The data context is formed of:

A static part, independent from the page

Formed of the user session data, that is of entities of type 'session' and of type 'action'.

A dynamic part, depending on the block tokens inserted in the page

Formed of the entities of type 'data access' created upon processing the block tokens.

Context management

Initial state

When the page construction starts, the data context contains only the session data.

Block processing

When the XTEND server processes a block it performs an iteration on all the selected entities.
For each entity, following processing is run:

1. Piling up the data (entity) in the data context
-->The entity is positioned at the top of the stack and is processed first upon definition of the value of a field.

2. Calls the HTML construction method for child tokens
-->This mechanism is recursive and makes it possible to pile up blocks

3. Unpiles the entity
-->After processing the child tokens on the server, unpiles the entity before processing the next one.

The entity is at the top of the stack and is addressed by the ACURRENT block.
For instance a selection criteria 'CODE=ACURRENT.CODE' in a dynamic link indicates that it is necessary to select the element of the current line

Definition of a field value

The principle of defining a field value consists in:

1. going through the entity stack from top to bottom

2. stopping when the read entity contains a field with the same name

If a field can be found, XTEND send the value of this field, otherwise, an empty value is sent.

The example below shows the stack situation with a mono-record background block (BLOCMONO) and two multi-record nested blocks (BLOCMULTI1, BLOCMULTI2).

The first multi block is positioned on the 2nd line and the second block (BLOC2) is positioned on the 3rd line.

<!adx="BLOCMONO">
    <!adx="BLOCMULTI1">
        <!adx="BLOCMULTI2">
            <span adx="MyField">
        <!adx="BLOCMULTI2">
    <!adx="BLOCMULTI1">
<!adx="BLOCMONO">

Stack	Order
ASESSION	3 - Bottom
ENTITE_BLOCMONO	2
ENTITE_BLOCMULTI1_LIGNE2	1
ENTITE_BLOCMULTI2_LIGNE3	0 - Top

In order to define the value of the MyField field, the XTEND server verifies the presence of the MyField field in the entities according to the following order:

1. ENTITE_BLOCMULTI2_LIGNE3

2. ENTITE_BLOCMULTI1_LIGNE2

3. ENTITE_BLOCMONO

4. ASESSION

The principle applied to define the value is the same for:

The field tokens
The setups/criteria of type 'Field token' (MyBlock.MyField) used in:

The selection criteria of blocks and dynamic links
The Web action setups
The criteria of the conditioned block formulas

In order to calculate the value of a field for a given entity, it is necessary to use the following syntaxes:

ASESSION.MyField

Reads the value in the session context

MyBlock.MyField

If MyBlock is a mono-record block, the field value for this block is read.
If MyBlock is a multi-record block, the field value for this block is read for the selected line

MyBlock(i).MyField

Reads the value of the field for the i-th line of the block (i beginning with 1)

Definition in session context

The session context is formed of the different data blocks described below.

The definition of the field value is made according to the presentation order of blocks in the list.

1. AUSERINF - Logged-in user information

Contains the data of the user account created upon login (see AXLOGCOD/AXLOGVAL)

2. AUSERVAR - User variables

Contains the fields created via the functions JavaScript xtdAjaxSetUserVar and xtdSetUserVar
Used to record the client data on the server

3. ASITE - Site document

Access to the values of the fields 'Free setups' of the site record
<span adx="ASESSION.MyFreeField"></span> displays the value of the free setup of code MyFreeField

4. AMESS - User messages

Contains the messages returned by X3

5. AUSERCRIT - User criteria

The selection criteria are saved and restored via the HTML xcrit setup
<input type="text" adx="MyCriteria:xcrit">

6. ASYSVAR - System variables

Contains only the ATODAY field

7. ACONST - Field tokens of type constant

See the field tokens

HTTP query processing

HTTP queries

In the X3WEB structure hosting the XTEND server, there are 3 application layers:

1. the front-end HTTP Apache server processing HTTP queries

2. the Tomcat Apache server hosting the 'Web applications'

3. the XTEND Web application

The HTTP queries to the XTEND Web application are identified by an URL which begins with /xtend/*.

The 'path' of the XTEND Web application can be set up via the console using the advanced setup:
xtend.server.virtualpath.context=xtend

The Web application is able to process a certain number of queries (servlets) identified by the 2nd setup of the URL /xtend/servlet/:

/xtend/page/* manages the HTML page construction
/xtend/data/* manages the access to file data (pictures, flash, pdf file...) on X3 or on local mode
/xtend/admin/* manages administration queries useful to the developer
/xtend/svc/* manages administration queries useful to the developer (log file, cache management, reload...)

GET or POST method

The HTTP protocol proposes different methods or commands specifying the type of action that the query carries out.

The XTEND server responds only to GET and POST which mean the following:

A dynamic link uses by default the POST method if a selection of action is associated to it.
This method submits the HTML form <form></form> of the page by calling the function JavaScript xtdDoDlk.
This function calculates the setups of the dynamic link (xml) and sends them to a hidden input field .
The server receives all the information it needs to process the action or selection associated with the dynamic link which was clicked on (data of the form fields and context calculated by the client).

A dynamic link can use the GET method if it only redirects to a page or if there is no action or selection setup to calculate.
This method does not submit the HTML form but redirects the browser to an URL containing only base setups (link, solution, folder, site, language).

Only the GET method is compatible with the search engines (web crawlers).

Using a 'reverse proxy'

For safety reasons, it is possible to use a reverse proxy to access applications from internal severs.

Using a reverse proxy makes it possible to hide the actual address of a server and makes site maintenance easier.

In the case of server stopping it is possible to redirect temporarily requests to another server without the client knowing.

In order to define a 'reverse proxy' with the alias proxyxtend in the Apache server of the X3WEB server hostproxy which "redirects" the XTEND requests to hostxtend, following configuration is used.

File Apache2.2\conf\httpd.conf

Create a 'virtual host' on the port 80

<VirtualHost _default_:80>
    <Location "/proxyxtend/">
         ProxyPass                                "http://hostxtend:28980/"
         ProxyPassReverse                     "http://hostxtend:28980/"
         ProxyPassReverseCookieDomain "hostxtend"   "hostproxy"
         ProxyPassReverseCookiePath       "/xtend"       "/proxyxtend/xtend"
    </Location>
</VirtualHost>

File Apache2.2\conf\extra\httpd-ssl.conf

<VirtualHost _default_:443>
...
    ProxyRequests Off
    SSLProxyEngine on
    ProxyPreserveHost On
    <Location "/proxyxtend/">
         ProxyPass                                "https://hostxtend:28943/"
         ProxyPassReverse                     "https://hostxtend:28943/"
         ProxyPassReverseCookieDomain "hostxtend"   "hostproxy"
         ProxyPassReverseCookiePath       "/xtend"      "/proxyxtend/xtend"
         SSLRequireSSL
    </Location>
...
</VirtualHost>

XTEND configuration

When the hostproxy Apache server receives the url http://hostproxy/proxyxtend/xtend/page/*.
It behaves as a proxy and carries out for the client the request http://hostxtend/xtend/page/* to the XTEND server.

The XTEND server which calculates the HTML page to send to the client must take the reverse proxy into account to generate URLs correct for the final client (the browser).

In our case, XTEND must generate URLs in /proxyxtend/xtend/*.
It must take the presence of the reverse proxy into account to calculate the URL.

The method used by XTEND is the following:

-1- Declaration of reverse proxies

Declaration of HTTP and HTTPS ports of 'reverse proxies' via the administration console by modifying the following advanced setups:

# Configuration reverse proxies
xtend.server.gensetup.proxies.hosts=      hostproxy1   hostproxy2
xtend.server.gensetup.proxies.portshttp= 80           80
xtend.server.gensetup.proxies.portshttps= 443          443

-2- Read of the HTTP 'Referer'

The HTTP Referer header contains the url used by the client to reach the server.

The XTEND server analyzes this URL and deduces that a reverse proxy is present if /xtend/ has a value as prefix (example /proxyxtend/xtend/).

The previous configuration is used to manage http->https protocol changes and conversely. The XTEND server must know the HTTP and HTTPS ports of the reverse proxy.

In order to work properly with a reverse proxy, the XTEND server must compulsorily know the HTTP 'Referer' and the HTTP and HTTPS ports of the (first) reverse proxy.

The configuration setup xtend.server.gensetup.http.askreferer=on/off (on by default) is used to activate or deactivate this mechanism.

The 'HTTP 'Referer' header is not always present, especially when entering the address in the browser address bar.

If the HTTP 'Referer' header is not present in the request, the XTEND server sends to the browser an empty form with automatic response (hidden to the user), which in most cases sends the 'Referer'.

If the attempt fails and the 'Referer' is not sent, the XTEND server cannot take into account the presence of a reverse proxy.

The session

The communication protocol between the Web browser and the HTTP server is the HTTP protocol which is "without report".
The HTTP server does not keep any trace of the client after request processing.
Everything goes as if the connection between the client and the server as cut after each request.

The functioning mode is radically different from the client/server mode (client X3), which guarantees an active connection with the X3 server during the whole session.

In order to develop Web applications, it is necessary to link the various requests to associate them with a same client.
This link is made by using the HTTP cookies.

A cookie is created upon initiative of the server when the latter sends to the client a 'set cookie' instruction to the HTTP header.
The cookie is then sent via the client to the HTPP header of all the requests to this server

Instruction
Set-Cookie: JSESSIONID=3C9B3EF39FB53069ACB02B6ADA5551A1; Path=/xtend
Path is used to filter the cookies sent to the server
In our example, JSESSIONID is sent to all the requests whose URL begins with /xtend/ (http://host:port/xtend/....)

The identification by cookie (and thus the functioning of the Web application) works only if the browser authorizes the cookies of type session (non saved).
This is generally the default setup of Web browsers

The notion of 'client' differs according to the browsers and depends on the way they manage cookies.

For Firefox, the cookies are shared between browser instances.
When logging in to a same XTEND site on the same workstation with two Firefox browsers, the latter do not have the same XTEND session.
The browsers are seen as for the browsers as a same client.
For IE6, the cookies are not shared between browser instances.
When logging in to a same XTEND site on the same workstation with two IE6 browsers, the latter have separate session.
The browsers are seen as for the browsers as two different clients.
Within a same browser, the tabs have the same cookies and are seen as a same client.

TOMCAT session

The user session used to associate a specific data context to each client is managed by the TOMCAT server via a servlet approach.

In order to identify the requests and redirect them to the right user session, the TOMCAT server manages a specific session cookie of code JSESSIONID.

Cookie managed by the TOMCAT server
Set-Cookie: JSESSIONID=3C9B3EF39FB53069ACB02B6ADA5551A1; Path=/xtend
JSESSIONID is the TOMCAT session identifier

The JSESSIONID cookie is not long-term, that is that it is not kept when closing the browser.
When closing the browser, the session and thus the data context is lost.

The session is kept in memory for a duration which can be set up using the console configuration setup.
xtend.server.gensetup.http.session.timeout
This setup indicates the inactivity time in minutes authorized by the TOMCAT server for a user session.
If no activity can be detected (request received) during this period, the session is deleted and the data context is lost.

Reconnection mechanism

The XTEND server manages its own session identifier via the XADXID cookie.
This cookie is long-term, that is that it is saved by the browser (if authorized).

When the user opens their browser and reconnects to the XTEND server, the XASXID cookie is sent in the request (if it exists) and the server tries to find the (TOMCAT) session of the user to reconnect.

Cookies managed by the XTEND server
Set-Cookie XADXID=123332746651556912893670; Expires=Sat, 30-Jan-2010 14:57:46 GMT; Path=/xtend

The console configuration setup xtend.server.gensetup.http.cookie.sess.persist=on (on by default) is used to activate/deactivate the management of the reconnection following the browser closing.

The reconnection page of the site (optional) is used to inform that the session has been restored.

Functioning without cookies

XTEND proposes a functioning mode without cookies which propagates the JSESSIONID session identifier in the URL.

This mode can be activated:

Either by the user via the ASESSSWITCHCOOKIES Web action or via the ADLKSWITCHCOOKIES dynamic link
By default for all the sessions via the console configuration setup
xtend.server.gensetup.http.cookie.disabled=on - off by default.

The mode without cookies increases the size of URLs by adding a setup:
http://ecfdalbopro:28980/xtend/page;jsessionid=E04AEF0615702B3C7E52107ED6C17D8A?DLK=DLKTESTMAPPING...

This mode does not allow the use of link to other Web sites because the session id is not propagated and the user session is lost.

Server processing

This paragraph describes the processing process of HTTP requests by the XTEND server.

Apache HTTP server

1. Receiving the HTTP request

Managing the HTTPS secured protocol

2. Filters the request depending on the URL

Only the URLs beginning with '/xtend/*' are processed by XTEND
Console setupxtend.server.virtualpath.context
This is the httpd.conf which contains the Apache server configuration
This file is generated when configuring the X3WEB server

3. Transferring the request to the 'Apache TOMCAT server' via the mod_jk
mod_jk : 'Jakarta module' is the name of the free software project which resulted in the TOMCAT project

JkMount xtend/portal* ajp13
JkMount xtend/page* ajp13
JkMount xtend/err/* ajp13
JkMount xtend/ajax/* ajp13
JkMount xtend/x3rsrc/* ajp13
JkMount xtend/htmrsrc/* ajp13
JkMount xtend/x_protect/* ajp13
JkMount xtend/data/* ajp13
JkMount xtend/svc/* ajp13
JkMount xtend/*.jsp ajp13

Apache TOMCAT server

1. Creates a task (thread) for the request processing

2. Finds the (servlet object service) identified by the URL

/xtend/page/* for the processing of HTML pages

3. Finds the session with the JSESSIONID cookie

Creates a new session if the session cannot be found

4. Calls the doPost or doGet method of the service

An httpRequest setup contains the data of the http request and the user session
The XTEND Web application processing begins here

XTEND Web application

1. Initialization

1. If new TOMCAT session

Reconnection processing via the XADXID cookie
Http referer request ('reverses proxies' taken into account)

2. If redirection (Protocol change, etc.)

URL redirection processing

3. XTEND setup read

GET method: URL setups
PUT method: XML setups in the XTDXML field of the HTML form
Setups: clicked dynamic link, requested page, action setups, etc.

2. Session opening

1. On hold of the session is used by another request

Setup console xtend.session.wait.timeout

2. Context positioning on the XTEND site

The code SolutionX3/DossierX3/SiteXtend is used as setup for each request
If there is not existing setup or site, the default site is the one chosen
Setups console xtend.server.gensetup.defsite.x3sol,x3fldr,xtdSite

3. Verification of the XTEND dictionary updates

Technical setups of the site 'Verify updates'

3. Controls

1. Protocol control

If the (http or https) protocol of the request is not the one defined for the page

Sending a redirection to the client (Http status=302) to change of protocol

2. If the user is logged in: control on the inactivity time of the XTEND session

Timeout session(mn)' setup of the site record

3. If the page is protected
Setup 'Protected access' of the page record

1. Control on the user access rights

Control depending on the profile if the user is logged in

2. If the user does not have the correct access rights

Redirection to the login page defined in the site record
Saving the context (action, destination page) to restore it after login.

4. Web action processing if an action is associated with the dynamic link

The action can modify the destination page
For instance if an error happens, the current page is still displayed, along with the X3 message

5. Page display

1. Verification of the page updates

Technical setups of the site 'Verify updates'

2. Initialization of block tokens (requests)

Calling Web services of type Data access

3. HTML calculation

Going through the token tree structure
Calling the method computeHtml

6. Sending the response to the browser