Macrotone Blogs

Macrotone blogs upon Joomla, our products and other matters.

An IT professional with a wide experience of IT systems, specialising in Database Management and Security.

Making a component truly Multilingual (6)

We duly released our ‘multilingual’ component Rialto, and there are just a few additional points worth mentioning before we wrap up our series of blog posts upon this topic

One cool trick that we discovered from reading the Joomla documentation is the ability load arbitrary front-end and back-end language files of any component and language. For example, you may want to load the English language file and mix it with the user's current language. This means that untranslated strings will appear in English and not as untranslated keys. You can use something like this for the back-end language files:

$language = JFactory::getLanguage();
$language->load('com_yourcomponentname', JPATH_ADMINISTRATOR, 'en-GB', true);
$language->load('com_yourcomponentname', JPATH_ADMINISTRATOR, null, true);

And to load the front-end language files, using the same trick:

$language = JFactory::getLanguage();
$language->load('com_yourcomponentname', JPATH_SITE, 'en-GB', true);
$language->load('com_yourcomponentname', JPATH_SITE, null, true);

This  is particularly useful for the situation where not all of the ‘foreign’ language translations are up to date, yet you still wish to present an ‘intelligent’ output.   

One associated use of the above is the situation where we have the front end making use of helper files from the back end, and the strings used in the helper files are (obviously) making use of the back end language files. To avoid re-specifying the strings in the front end language files we can ensure that the back end files are picked up by including the above in our helper files.

We have avoided describing how a multilingual site is set up, as this is adequately covered in many other posts upon the net. We would again strongly recommend the component named ‘JAMultiLingual’ from Joomla Arts that we have successfully used to set up a Joomla site for ‘Multilingual’ and certainly simplifies the steps in getting started into Joomla Multilingual. It is not the only possibility but one which we have used and found very well constructed.

Issue Tracker: Handling received emailed issue updates

issues-48Our Issue Tracker component for Joomla has long had the ability (when configured) to handle ‘issues’ sent to a predetermined email address upon a mail server, and to automatically create issues. What is not quite so well known is its ability to handle updates to existing issues via received emails as well.

Certain criteria have to be established to ensure that the ‘update email’ relates to an existing reported issue.  There are several steps to this.

a) When an issue creation notification is send out the sent email contains a couple of ‘special’ mail header tags.  When an update email is received the presence (or absence) of these mail header tags are checked, and provide one level of confirmation that the ‘email update’ is for an existing issue.

b) The email title is also checked for the existence of a special format string.  The format of this string is usually [Issue: xxxxxxxxxx] or just [xxxxxxxxxx], where xxxxxxxxxx is the issue number. It is possible to have different formats for the issue number which may be a ‘random’ ten character string, or a nine digit number preceded by zeros and a leading character (again ten characters in total), or a numeric string consisting of up to 10 digits.

[Release 1.6.8 of Issue Tracker enables a slightly more relaxed format string in that any character string may precede the issue number. The square braces are required in all cases.]

We can assist in the issue number detection by specifying one of the above issue format strings in the outgoing message notification. This is a component configuration and the format of message header and message body contents is possible for all outgoing message types.

c) The email senders address is also checked to see whether it matches the email address of the person who first raised the issue report.

If the criteria is not met then the email update is rejected.  If however the email is identified as being an update for an existing issue, then the handling of the email body has to be looked at.  A number of email clients (all?) such as Outlook, Thunderbird etc. (to name just a couple) permit ‘reply’ emails to incorporate the received email text in the outgoing message either preceding or following the ‘new’ message text. This is potentially annoying in that if the body text was taken as it is received, the message would contain unrequired information, which is already present in the issue ‘record’. We therefore seek to eliminate the ‘former unrequired included text’ from the message before saving the data in the issue record.

The easiest way to be specify a special ‘text string’ which is searched for in the message body and any text seen after this ‘string’ is silently ignored.  This works well in practise but only if the ‘string’ is present in the message.  If it is not present one is left with a decision as to whether to accept the whole message, ‘included former messages text’ or just reject the message entirely.  This is obviously a decision that a configuration option permits a site to make.

It is not possible to know whether the client browser, is going to include the ‘former message text’ either before or after the ‘new’ message content,and there is no standard way for a client to indicate the present of this ‘former’ text in the message.  Fortunately the most common default is for the ‘former message’ to be included after the ‘new message text’. In this situation we can try and eliminate most of this text automatically.  To do this we need to ensure that the outgoing message contains our ‘'special reply text string”. We can accomplish this by the use of a ‘hidden’ HTML tag at the start of the outgoing message as follows:

#-#- Text following text is ignored –#-#

 [The ‘#-#-‘ string (and its derived inverse) is a component setting and can be any desired unique string that is unlikely to be provided by the user in the usual course of business.]

We add this to the email template body and it has the advantage that it will not be displayed in the message due to the span style which hides the text. If the user replies and includes the received message contents in their reply it would be picked up by the message body checking code and can then be eliminated from the message stored in the component issue.

This will not of course eliminate all unnecessary text from the ‘email update message’ but it can go a long way towards reducing it. 

Making a component truly multilingual (5)

Whilst finishing up our component there were a few matters which we needed to address.

 

Site Map Generation

The first of these related to the generated ‘site map’ which was not displaying correctly. The problem was caused by us omitting to include the ‘new’ alternative language main menu items in our site map generation component (OSMAP/XMAP) which soon resolved the matter. To be more precise the ‘site generator component’ was installed prior to our multilingual work upon the site, so did not ‘automatically’ pick up the menu items it uses in the map generation. Obvious once one looked at it. It did reveal an addition place where we had to add out additional ‘'database query condition to include only the current displayed language (plus those marked all or undefined).   This just required adding to the ‘getCategories’ method, although to be absolutely certain that only the relevant entries were shown, we also added it to the ‘GetEntries’ method as well. This latter is probably not required except where entries are added to the incorrect categories, but acts as an extra check when the entries are generated.

Deletion of Associated Items

This actually applies to both the front end and the back end of the site. We decided to take a deeper look at the code to handle the deletion of any associations when an item is deleted. To be more precise when an item is deleted, the existing association entry for that item should also be deleted from the associations table. As originally written all association records would be removed for the deleted item and hence also for the associated items. However this is not necessary correct when there are three (3) or more associated items, since one really only wished to remove one association record.  For two associated items it is fine, since we also wish to remove the ‘other’ record in the associations table at the same time. .  So to be precise, we should for situations where there are more than 2 associations for an item, only remove the single record in the associations table, but if there are only two item, then we should remove both records from the associations table. This has worked well in all our tests and wonder why some similar code does not exists in the base Joomla components such as com_content etc. On a busy site with items being created and deleted on a regular basis we can see the situation where the database associations table would be much larger than is necessary, since there would be without similar code being present in the components, resulting in a slower response to other queries.

 

How the PHP tests work

One of the things we omitted to mention in our last post was how the actual code checks were working and didn’t explain what the checks were actually ‘checking’. We made use of two specific methods which we expand upon below:

JLanguageMultilang::isEnabled()

This method determines if the language filter plugin is enabled.  It works for both site and administrator on the site.  If the site is supporting multiple languages then true is return else false is returned.  The presence of the language filter plugin is what is determining whether multiple languages are being used upon the site. Note it does not check whether multiple languages are actually installed.  The distinction is subtle but very clear.

JLanguageAssociations::isEnabled ()

This method is used to determine if the language filter Items Associations parameter is enabled.  One can always have a multilingual site and not make use of item associations, which is ‘relating’ items that are ‘effectively’ the same, but only expressed in a different language. An example might be a site located in a country with two different locally spoken languages (i.e. Serbia) and items are translated and presented in both languages to site visitors.

Making use of the two standard Joomla provided methods simplified our coding, however we still had to add our own code to pick up any desired associations for our displays.

 

To be continued…

Making a component truly Multilingual (4)

Continuing on from our earlier posts about item associations. we continue our look at the front end.  As explained this is slightly more complicated.

We require changes to the following files:

There is the module in place required by JMultilingual that performs the ‘switch’ between the various installed available languages. We need to pick up the current displayed language and modify the displayed data based upon the determined value.

We need to implement the following changes:

Models that display lists such that there is an additional statement in the SQL query to detect the language. Such a statement might look similar to the this:

        $query .= " WHERE c.language in (" . $db->quote(JFactory::getLanguage()->getTag()) . "," . $db->quote('*') . ",'')";

This will pick up all the current displayed items for the determined language and also any items that are marked as ‘All’ or undefined. Hopefully there will be no undefined items but this allows for that possibility.

The front end views need modifying to display the ‘language’ field if deemed necessary.  We need to add the ‘language’ field to any select statement so that it is then available.

Any drop down select lists (if present) similarly need modifying so that the ‘language’ criteria is also applied. i.e. If being viewed in Spanish, we do not want to display any drop down list selectable item in English. The appropriate entries would have been defined in the admin side of the site, along with the required associations.

Any list view models will probably need to have some additional criteria added to the select statement along the following lines:

if ( JLanguageMultilang::isEnabled() ) {
if (JLanguageAssociations::isEnabled() ) {
// Criteria where associations will change the displayed values.       // …    } else {       // Any criteria where associations are not used.
      // ……    } } else { // Criteria for non multilingual sites.    // Effectively the code we had before we implemented multilingual. }

Any displayed module using the component data would similarly require the language criteria to be added to any select statement. Whilst on the subject of modules it is worth mentioning that any link which might be present in the module, to display more specific data would need inspection to ensure that they still work. We found that if a menu id (itemid) is specified in the link that it did not always result in the correct data being displayed.

If we are allowing items to be deleted in the front end then we also have to modify the appropriate model delete code to handle any ‘associations’ that may exists for the item to be deleted. We can choose to use similar code to that used in the back end.

The creation of item associations in the front end was touched upon in our last post upon the subject. Further investigation reveals that there was a conscious decision by the Joomla Multilingual team not to implement associations on the various ‘base’ Joomla components, instead relying on the back end functionality to perform the necessary actions. In the lack of more information at the current time we have decided to adopt the same approach, but may change this at some later time..

To be continued…

Withdrawal of Support for Joomla 2.5

With the removal of official support for Joomla 2.5 by the Joomla Project, we regret we are no longer able to provide general support for our Joomla components on this version.  Instead our support will concentrate upon the latest Joomla 3.x versions, currently 3.4.

Should you experience with our extensions upon Joomla 2.5 we will try to help and advice and possibly investigate but no new features or enhancements will be developed.

We recommend our customers migrate to the latest officially supported Joomla version as soon as it is possible.

Making a component truly Multilingual (3)

Following on from our earlier post about item associations. we next turn our attention to look at the front end.  This tends to be more complex since it is here that we are actually using the associations to impact the front end displays.  To expand upon this, when a user chooses to display the categories in for example ‘German’ it is the ‘de-DE’ categories that should be displayed, along with the appropriate ‘de-DE’ translated text.  Likewise if they then choose to display ‘English’ the ‘en-GB’ categories are displayed along with the English text.  It is also reasonable to expect that any categories that are marked as ‘All’ languages would also be displayed in each case.

The specific ‘category items’ to be displayed for a chosen language would be the ‘specific’ language items, together with the items marked as ‘All’' languages’. The question of what about the un-translated items which are not associated with a ‘specific language version has also to be considered.  This is best explained by considering a specific situation.  Assume that we have our ‘Rialto’ component available with a English (en-GB) and a German (de-DE) translation.  All of the categories are established with the correct associations, however they may well be a number of different ‘entries’ that are present in only one specific language, which would not then have any associational entries in the other language.  In our Rialto example, the entries are ‘classified advertisements’, it is not unreasonable to assume that an advert for an item to buy/sell in German, may not be of interest to an English person, and that might therefore not be a translated version of the advert created. The question is then does one display the ‘German’ item to an English viewer?

We are assuming here that the ‘English’ viewer is displaying the site in the English language, and that a ‘German’ viewer is displaying the site in the ‘German’ language.

Taking this one stage further, assume that we wish to determine the total number of entries in a specific category. On the surface this sounds simple, but once we are multilingual we have  a few additional complications. First we have the number of entries for which there is a single language specified (or all) for each of the different associated categories. This means that all the un-associated de-DE German entries in a category are counted together with the number of un-associated entries in the equivalent en-GB category are to be counted. Then we have the entries where we have an entry associated, in this case we only wish to count the entry once, even though there may be a de-DE and a en-GB translation they are effectively the same entry for counting purposes. This makes getting the total number of entries complex in terms of the SQL statement that is required, especially given the underlying table structure, involving sub-queries ( or numerous joins).

The approach we have taken is that we would only, by default display specific items in the selected ‘viewing’ language (plus items marked as being ‘all languages’).  Switching the site language to one of the other possibilities would (probably) display different entries. i.e. This marked as ‘all’ and those available in the current selected language whether associated or not).

This might seem complex but should provide the site visitor with the entries in the chosen site language and those marked as visible for ‘all’ languages.

There is also the question of which entries are displayed in the ‘module’ which displays the latest ‘n’ (where ‘n’ is the number) of entries recently opened/created.  The entries displayed are interpreted as being those created/opened in the displayed site language and those created/opened as being ‘all’ languages.

Similar to the changes for the back end we have to modify some of our code but the changes are different, mainly because it is only really the actual items created/maintained upon the front end that need the most work.  We require changes to any ‘drop down’ selection boxes so that the displayed options are in the chosen site display language, which means changes to the ‘helper’ files. Other changes relate to the displays which are themselves more involved than for the back end, since we can create entries on the front end and (presumably) also need to probably create entry associations as well. This last task is something that, from our study code reading is not something that Joomla com_content does currently so may well be something that we ourselves put on the back burner, leaving the back end to create the associations between entries. Not ideal perhaps but it seems to be something that Joomla multilingual sites are currently having to wrestle with already, so will be familiar to them.

To be continued…

Making a component truly Multilingual (2)

In an earlier post we mentioned the topic of associations. In Joomla terms associations are effectively links between equivalent items that are in different languages. In our component we also wish to allow associations so there are a few changes we need to implement for this as well. There are no table changes required since we will be making use of the supplied Joomla associations table.  This table is composed of three columns, the first being an id column that is the value of the specific item being referenced. In our case it is the id of the category or entry. It is worth mentioning that within the Joomla associations table the id column is NOT a primary key. The second column is a context value, which is typically composed of the name of the component followed by the specific component type separated by a full stop. i.e. ‘com_rialto.category’ or ‘com_rialto.entry’.  The third and final column is a hash (md5) key. This key is what is used to ‘associate’ the various parts together. So as an example two categories associated together would have the same ‘hash key’.

Associations – Back End

The setting up of associations is something that seems to be restricted mainly to the back end so it is these changes that we will look at first.

The following changes are required:

  • Modify the views such that if associations are enabled they are displayed as required.
  • Modify the model save method so that upon the item being saved the existing associations are removed and ‘new’ associations created.
  • Modify the model ‘delete’ method so that when a item is deleted any existing associations are removed. If the component is making use of the ‘trash’, i.e. when an item is deleted it is not actually removed, just relocated to a ‘trash’ status for removed when the ‘trash is emptied’ this also has to be allowed for in our code.   [Note that we found a problem in this area, in that the code we were following for the Joomla content component, there was no appropriate code to remove associations in the delete method. We think this is a definite Joomla bug, unless we have missed something we do not understand.]
  • Modify the model ‘get’ methods so that the associations for the specific item are picked up correctly. 

The change for the views is slightly more complex than it might initially seem, since there has to be incorporated a means for the selection of ‘other’ items for the associations to be made. This is achieved by making use of slightly modified modal views which are displayed to enable the selection to be made.  We say ‘modified’ since for a selection criteria one does not necessarily need to display all of the item details, only sufficient to enable a clear distinction to be made to enable the correct item selection to be made.  The ‘code’ we were following/studying also permitted editing of the item from these modal views, however we question the logic as to why one would wish or want to provide additional editing at this point. The coding is not difficult but just, to our mind, seems unnecessary.

This does effectively complete all of the back end changes with very little pain being encountered on the way.

The question of how the ‘entries’ are displayed using the associations is something that is more specific to the front end. In the back end we tend to concentrate upon the maintenance aspect of the various associations so the coding in the models is virtually unchanged as we wish to see ‘all’ of the entries and categories, irrespective of what ever language they may be written in. This is something we will consider in the next post in the series as there are a number of situations to the covered.

To be continued…..

Tags:

PHP Storm 9 available

We have been using the JetBrains PhpStorm application in our development for some years and knowing some of our readers also are interested we note that PhpStorm 9 is now finally released!

PhpStorm 9, the next big release of our IDE for PHP and web development, is already here. Make faster progress with PhpStorm 9’s refined editing and debugging experience, deeper code understanding, advanced productivity boosts, and complete support for remote development:

PHP Language & Editing Experience: Includes postfix code completion for PHP, partial PHP 7 support, advanced code understanding (Introduce parameter refactoring, advanced PHP type inference, support for persistent language injections, and new inspections and intentions), and other enhancements.

New Debugging Experience: Inline debugger for PHP and debugging workflow enhancements (php.ini can be opened from settings, and debugger config validation has been enhanced).

Remote Development: Remote edit, PHP Code Sniffer and Mess Detector via remote PHP interpreters.

Many improvements related to frameworks and tools.

A range of new and updated third-party plugins available in the PhpStorm plugin repository.

Making a component truly Multilingual (1)

One of the first requests we received for our Issue Tracker component after its initial release was to make if ‘Multilingual’.  At the time we were rather busy but recorded the request with the intent to implement it later when circumstances permitted.. We recently decided to look at the changes involved in more detail but decided to look at our Rialto component, which is not so complex and would (should) be easier to modify.

It is important to clarify exactly what is meant by a component being ‘Multilingual’.  It is one thing to have language translations available for a component that can be loaded upon a site to permit the component to display the inbuilt ‘text strings;’ in the native language of the site, but this does not in itself make the component ‘Multilingual’.  Perhaps the best definition is that a site can be considered ‘Multilingual’ if it is possible to support more than one language such that one can switch from one to another via a single mouse click.  Items like site categories, menu items, and articles etc. would at the same time change to be in the selected language. A component would be ‘Multilingual’ if it can itself support such transitions. The classic Joomla component that does this is of course the ‘com_content’ component.

There is one other aspect that needs to be mentioned and this is the matter of ‘Associations’.  Joomla permits an article, or a category (amongst other things) to have an assigned language.  If one were to translate an article into another language then the site would have two (or more) identical articles each the same apart from the specific language. Joomla then permits one to ‘associate’ the one article with the other, such that the ‘seamless’ change on changing the language in use is switched is achieved.

Of course one does not necessarily have to translate all articles, and it is possible to define a default language (indicated by a ‘*’) which is display which ever site language is in use.

Our Rialto component, which is a Classified Advertisement component, is a situation where it is expected that a ‘Multilingual’ site would possibly have categories in different languages, which are probably ‘associated’ but that the specific adverts themselves (known as entries) are unlikely to be present in anything other than the one language. This does not mean that advert entries might not be in multiple languages just that the situation is not likely to be so common.

We shall describe some of the changes that were required to achieve what we desired, but given the number of changes required will split it into several parts. We will endeavour to avoid ‘too’ much specific code, and instead concentrate on the areas where we made our changes. We will also describe a few ‘opportunities’ for enhancement that we discovered whilst implementing the changes.

Just before we start into the detail it is worth mentioning that there is a component name ‘JAMultiLingual’ from Joomla Arts that can be used to set up a Joomla site for ‘Multilingual’ and certainly aids in getting started into Joomla Multilingual. It is not the only possibility but one which we have used and found very well constructed.

Language Field

The first step is to add the additional ‘language’ field to the database tables that we wish to have language contained within.

ALTER TABLE `#__rialto_categories` ADD `language` CHAR(7) NOT NULL COMMENT 'Language code';

Obviously we require the same column present upon all tables not just the one indicated. We also have to consider whether to apply a default language setting, possibly ‘All’ (*) for any existing entries that may be present in the database.

Having made the table change we then have to make the following code changes:

  • Change the component list and single item views so that the field is displayed.
  • Modify the Model ‘getItem’ (getItems) methods to return the additional language field to the views
  • Modify the Model Item ‘Save’ method so that the language field is updated when an item is updated or created. Ensure that a suitable default is applied if required
  • Note that one has to allow for the option that the component is being used upon non-multilingual sites and thus ensure that all situations are allowed for.  There are some standard Joomla language routines that enable one to test whether languages are installed (Method JLanguageMultilang::isEnabled() ), which aid in the coding.

These changes are ‘required’ for the back end, but may also be required in the front end if appropriate. It is not common to permit the creation (or editing) of Joomla categories in the front end, but it one permits the creation of ‘entries’ (articles)  in the front end, which is true for the Rialto component then we have to modify these as well. It is reasonable to assume that in the front end of the site, if a person is creating a new ‘entry’ that the language currently displayed on the site is the one in which they are entering the entry details. Otherwise one has to provide the user with the option of specifying the language they are using. 

These changes are required for all tables.

To be continued…..

IP Mapping, clustering, refresh – performance impacts

ipmappingWe mentioned the other day about using HTML5 to determine a site visitors location and using it to be displayed upon a Google map. We are here looking at the various options that need to be considered when setting the parameters for the best ‘site impression’.  I have deliberately used the term ‘impression’ because most of the ‘work’ is actually performed by Javascript in the users browser. Each user will most likely be using a different machine, i.e. Desktop, laptop, tablet, smartphone etc., not to mention the different processors included in each type, and all these will different performance characteristics.  This each user’s impression of ‘how fast’ a site actually is, will be different, and this is without considering the impact of any network performance in a) obtaining the source data from the web site and b) the transfer of data to/from the Google mapping servers.

Continue reading
Go To Top

Joomla! Debug Console

Session

Profile Information

Memory Usage

Database Queries