Macrotone Blogs

Macrotone blogs upon Joomla, our products and other matters.

Joomla 3.4.x and Mootools

joomlaWe have long been aware that the use of Mootools in Joomla extensions is being discouraged, but also know that it is used in a number of different places in the core code and would be automatically loaded if required.

What we were not expecting was that relatively new extensions would be using it.  We have just spent the best part of a week playing with a particular problem involving Mootools that we thought we would share.  We tended to make things more difficult for ourselves as we were using a plugin to automatically disable Mootools (for a much older problem), which tended to ‘mask’ possible solutions which meant we would have reached a conclusion much earlier if we had remembered.

We were working on an update to a front end form and observed that calendar fields were not displaying the ‘popup’ calendar picker window, also that when displaying the ‘tags’ field to enable the user to select or create new tags to add to an item, that the displayed field did not match what we were expecting, or seeing in the back end of the sight using the same code constructs.   We could see a Javascript error in the error console that was either a) Tips not found, or b) Squeezebox not found.  The first coming from a form field tool tip, and the second coming from the form calendar field.  These were obviously related to the form elements, but the tag display didn’t appear to have either of these used in its construction.

The tag display with Mootools disabled looks as follows:

20151221154101 image 4

We tried adding in explicit call to load various Mootools elements yet were not seeing any change in the problems.  After much thought and investigation it was then that we realised that we had the Mootools disabler plugin enabled so all our changes were effectively being ‘ignored’.  After a little silent ‘blue’ language we disabled the plugin and the Javascript errors disappeared and the calendar popup was visible again. 

What we had not expected was that the Tag display would also correct itself, which effectively meant that the Joomla tags helper must be using Mootools somewhere in its code. 

The desired tag display form field required (and achieved):

image

Despite searching we couldn’t find any reference that documented this, so its one to watch out for if you meet it yourselves.

Rialto v1.2.0 released

rialtoWe are pleased to announce release 1.2.0 of the Classified Ads component for Joomla named Rialto.

This component runs on Joomla 3.4 (and above) only. 

This release adds two major features, the first is support for Joomla Tags.  The second is that the front end Entry edit now makes use of JForms for better and easier control. One other significant change is the implementation of Joomla ACL controls, although this release does not fully implement it in the front end of the site. This will be more fully used in future releases. There are a number of minor changes/fixes also implemented in the release.

See the change log and documentation for more details.

The release is available in the download area.

Tracking down CSS icon problems

We recently noticed a strange effect upon our web site, when we implemented a new component which was making use of the Joomla Searchtools and the JHtml 'jgrid.published' method upon the front end.  The basic problem was that the methods which use various icons such as icon-search, icon-publish icon-unpublish etc. was not displaying the icons and instead all we were observing was a small square rectangle where the icon should have been shown.

It was apparent that it had to be a site template ‘feature’ since when using the Joomla Protostar template all icons were correctly displayed.

We have made use of a Bootstrap 3 based template for some time and not observed this effect before, which is probably because we had not tried to use these Joomla methods before in the site front end.

Using the Firefox Web Developer plugin, we could see the CSS that was generated, and we could try and edit the CSS until we could achieve  the desired result, which was obviously to see the icons. What puzzled us for a little while was why after adding what we thought were the correct CSS commands the screen display (Firefox) didn’t immediately show the icon specified, and instead displayed a small lined box icon.  This made us immediately think that we had the wrong code, but after adding the same codes to the actual template and refreshing the page, clearing the cache etc., the requested icons did indeed appear.  Whether this is a Web Developer feature is unclear but it was certainly misleading!

The Bootstrap 3 template makes use of Awesome Fonts and it appears that something or somehow the font family has been disassociated from the icon class, so we added a few additional CSS rules to the template and this resolved the problem.

.icon-publish:before, .icon-save:before, .icon-ok:before, .icon-checkmark:before {
font-family: FontAwesome; content: "\f00c";
}
.icon-unpublish:before, .icon-cancel:before {
font-family: FontAwesome; content: "\f00d";
}
.icon-trash:before { font-family: FontAwesome; content: "\f1f8"; }
.icon-search:before { font-family: FontAwesome;    content: "\f002"; }
.icon-archive:before { font-family: FontAwesome;    content: "\f187"; }

The above FontAwesome content codes are for the 4.4.0 release.

Whilst investigating this we also looked at a problem that we has observed some time ago but never resolved, which was that the small drop down that we present on some pages which contains the icons to Print or Email the page contents, was difficult to see, and only upon hovering with the mouse over the small box where the dropdown was positioned, actually revealed its presence.  We wished the presence of the box to be more visually evident.

Again Firefox Web Development Tools helped and we could see that the dropdown CSS code was effectively empty so we had to specify a background colour as below:

a.btn.dropdown-toggle
{
background-color: #99ccff; 
}

This added a nice light blue colour to our background and achieved what we desired.

We are still a little puzzled as to why the above was required, but suspect that the reason is that Joomla itself has implemented specific changes to its own Bootstrap 2 implementation as used by the Protostar template, as since we were/are using a template based upon Bootstrap 3 we need to add these additional CSS codes to ensure functionality is maintained. This tends to support our conclusion from an earlier problem with the use of vertical tabs, which are not supported in Bootstrap 3, but that is a different post topic completely!

Tags:

Kbase 1.1.0 released.

l com kbase
We are pleased to announce the availability of Kbase release 1.1.0 for Joomla 3.4

Kbase is a 'knowledge base' component used to present support information for our developed components in a user friendly manner upon our web site.  This update corrects a few very minor language problems and adds a new Glossary (Acronyms)) feature.

Updated documentation is available both as web pages and as a downloadable PDF file.

The update is available in the download area.

Perseverance wins the day!

We recently investigated a problem upon our web site that impacted our Forum, and the ability to submit new topics or edit 'old' topics.
The problem was complicated in that there were no error or warning messages to provide any indications of what may be the cause. The most obvious symptom was that all of the problems were related and concerned the inability to submit 'form' data. i.e. The login process submits the entered details to the login process.  The creation or editing of a forum post involves submitting 'entered data' to the post process.  

Our immediate thought was that we were seeing a jQuery conflict, since we have seen similar problems in the past, again with no indication of the cause. We then proceeded to look at all the components, modules and plugins that we use upon the site but despite intensive searching we couldn't find anything that might be the cause, and there was certainly only one version of jQuery being loaded.  Investigation also revealed that only forms submitted by the forum software were impacted.  Very strange and seemed to indicate the forum software itself was at fault, but nobody had reported such a problem and when we tried it on another web site it worked perfectly.  We also tried installing an 'older' version of the forum software and this didn't resolve the problem either.

One complication was that we were also having problems with our backup software, and found that it was not possible to restore a backup to enable to perform some tests upon a copy of the live site.  We tried the software upon an earlier version of the site and it was working perfectly. This just made the problem more difficult to understand since that particular site was more likely to have a problem since it was more complex.

Once we have a solution to enable us to restore a current backup working correctly we performed a restore.  This was interesting since the 'restore site' worked perfectly and didn't exhibit the problems seen on the 'live site'.  This meant that the problem had to be caused by either a) the operating software upon the two platforms, b) the version of the underlying database, or c) the version of the PHP software used by the site.  Looking at the technical requirements of the software all were being met but there had to be something getting in the way.

Upon a hosted platform there is very little if anything one can do about the operating system, so we were left with the PHP and/or database versions. We looked at the PHP versions and tried a number of versions including 5.4.44, 5.5.28 and 5.6.12, without any change in the problem. Changing the database version is more complex since it required the creation of a new database and a restore.  The database versions upon both system was virtually the same being MySQL upon the live site and MariaDb upon the test site, although they were the same versions.

We therefore took the decision to try a system restore upon the live site with very little hope of a solution. To our surprise a system restore corrected the problem, which leads one to suspect that there was some corruption in the file system somewhere, but we currently have no idea exactly where at the moment. The most likely is that one or more of the forum software files was corrupt.

This was a long and exhausting investigation covering many days and resulting in delay to other activities.  Looking back if our backups had been working correctly then we might have got there sooner, which makes one ask just how regularly does one test the backups.  Obviously not enough has to be the answer in our case, even though we do test them periodically.  I suspect we are not alone in not testing the backups sufficiently often.

Tags:

MariaDB and Joomla update

Further to our original post, we have now carried out a series of tests and are pleased to say that we have found no problems with using MariaDB with Joomla.  Our tests compared version 5.5_44 of Maria DB and MySQL and to all intents and purposes no difference is noticeable.  We cannot say there is any difference performance wise, since we were testing upon different server instances, which makes such a comparison impossible to determine.

We can also say that our NAS in now running MariaDB as standard, so we will be using this version (5.5.44) for all our development work henceforth.

“For all practical purposes, MariaDB is a binary drop in replacement of the same MySQL version (for example MySQL 5.1 -> MariaDB 5.1, MariaDB 5.2 & MariaDB 5.3 are compatible. MySQL 5.5 is compatible with MariaDB 5.5). What this means is that:

  • Data and table definition files (.frm) files are binary compatible.
  • All client APIs, protocols and structs are identical.
  • All filenames, binaries, paths, ports, sockets, and etc... should be the same.
  • All MySQL connectors (PHP, Perl, Python, Java, .NET, MyODBC, Ruby, MySQL C connector etc) work unchanged with MariaDB.
  • The mysql-client package also works with MariaDB server.
  • The shared client library is binary compatible with MySQL's client library. “
Tags:

Joomla Smart Search Index size observations.

We have recently upgraded the underlying database version of our Joomla site to MySQL 5.5, from 5.1 and carefully observed the sizes of the underlying databases as we progressed. The source database was reported as being some 200Mb in size, which is not excessive for our modest site, but we were very surprised when after restoring the system to the new database the database size was reported as 37Mb.  We understand how databases store their data, so it was not the fact that it was smaller than was the surprise more the dramatic difference.

We then proceeded to rebuild the ‘Smart Search (Finder)’ indexes and checked the database size yet again.  This is necessary, since the backup does not by default include the Smart Search table data in the dump file. It was now reported as being 147Mb, and increase of 110Mb.

The fact that this increase was on our modest site, must make one wonder whether the benefits out weight the database storage overhead.  Ours’ is certainly not a large site, and the use of ‘smart search’ on large sites is not something we would wish to consider, and a dedicated standalone search engine, such as Solr, or possibly the Google Site Search, rather than the pure-PHP implementation used in Smart Search may well be found to be more suitable.

Smart Search itself consists of a number of ‘tables’ in the Joomla installation, of which at least 16  are "mapping tables". These mapping tables resolve the many-to-many relationship between search terms and the content items that contain them. In theory those 16 tables could be merged into a single table and doing so would have negligible impact on small sites. However, on larger sites there would be a substantial impact on performance because the mapping table would have a very large number of entries and updating such a large table can be time-consuming.  Maintaining the search term index requires a fair amount of disk space, which increases the more terms that content items contain. Since phrases are also indexed, the longer the phrases, the more disk space required. For Smart Search a reasonable compromise has to be reached between search usability and disk space usage.

Our observations must question this compromise and possibly we need to review the settings that we are actually using. It is likely that most readers may not be aware of exactly how large the ‘smart search tables’’ are in their own Joomla installations.  The decision as to whether to continue using the Smart Search’ component should also be reviewed.

Making a component truly Multilingual (6)

We duly released our ‘multilingual’ component Rialto, and there are just a few additional points worth mentioning before we wrap up our series of blog posts upon this topic

One cool trick that we discovered from reading the Joomla documentation is the ability load arbitrary front-end and back-end language files of any component and language. For example, you may want to load the English language file and mix it with the user's current language. This means that untranslated strings will appear in English and not as untranslated keys. You can use something like this for the back-end language files:

$language = JFactory::getLanguage();
$language->load('com_yourcomponentname', JPATH_ADMINISTRATOR, 'en-GB', true);
$language->load('com_yourcomponentname', JPATH_ADMINISTRATOR, null, true);

And to load the front-end language files, using the same trick:

$language = JFactory::getLanguage();
$language->load('com_yourcomponentname', JPATH_SITE, 'en-GB', true);
$language->load('com_yourcomponentname', JPATH_SITE, null, true);

This  is particularly useful for the situation where not all of the ‘foreign’ language translations are up to date, yet you still wish to present an ‘intelligent’ output.   

One associated use of the above is the situation where we have the front end making use of helper files from the back end, and the strings used in the helper files are (obviously) making use of the back end language files. To avoid re-specifying the strings in the front end language files we can ensure that the back end files are picked up by including the above in our helper files.

We have avoided describing how a multilingual site is set up, as this is adequately covered in many other posts upon the net. We would again strongly recommend the component named ‘JAMultiLingual’ from Joomla Arts that we have successfully used to set up a Joomla site for ‘Multilingual’ and certainly simplifies the steps in getting started into Joomla Multilingual. It is not the only possibility but one which we have used and found very well constructed.

Issue Tracker: Handling received emailed issue updates

issues-48Our Issue Tracker component for Joomla has long had the ability (when configured) to handle ‘issues’ sent to a predetermined email address upon a mail server, and to automatically create issues. What is not quite so well known is its ability to handle updates to existing issues via received emails as well.

Certain criteria have to be established to ensure that the ‘update email’ relates to an existing reported issue.  There are several steps to this.

a) When an issue creation notification is send out the sent email contains a couple of ‘special’ mail header tags.  When an update email is received the presence (or absence) of these mail header tags are checked, and provide one level of confirmation that the ‘email update’ is for an existing issue.

b) The email title is also checked for the existence of a special format string.  The format of this string is usually [Issue: xxxxxxxxxx] or just [xxxxxxxxxx], where xxxxxxxxxx is the issue number. It is possible to have different formats for the issue number which may be a ‘random’ ten character string, or a nine digit number preceded by zeros and a leading character (again ten characters in total), or a numeric string consisting of up to 10 digits.

[Release 1.6.8 of Issue Tracker enables a slightly more relaxed format string in that any character string may precede the issue number. The square braces are required in all cases.]

We can assist in the issue number detection by specifying one of the above issue format strings in the outgoing message notification. This is a component configuration and the format of message header and message body contents is possible for all outgoing message types.

c) The email senders address is also checked to see whether it matches the email address of the person who first raised the issue report.

If the criteria is not met then the email update is rejected.  If however the email is identified as being an update for an existing issue, then the handling of the email body has to be looked at.  A number of email clients (all?) such as Outlook, Thunderbird etc. (to name just a couple) permit ‘reply’ emails to incorporate the received email text in the outgoing message either preceding or following the ‘new’ message text. This is potentially annoying in that if the body text was taken as it is received, the message would contain unrequired information, which is already present in the issue ‘record’. We therefore seek to eliminate the ‘former unrequired included text’ from the message before saving the data in the issue record.

The easiest way to be specify a special ‘text string’ which is searched for in the message body and any text seen after this ‘string’ is silently ignored.  This works well in practise but only if the ‘string’ is present in the message.  If it is not present one is left with a decision as to whether to accept the whole message, ‘included former messages text’ or just reject the message entirely.  This is obviously a decision that a configuration option permits a site to make.

It is not possible to know whether the client browser, is going to include the ‘former message text’ either before or after the ‘new’ message content,and there is no standard way for a client to indicate the present of this ‘former’ text in the message.  Fortunately the most common default is for the ‘former message’ to be included after the ‘new message text’. In this situation we can try and eliminate most of this text automatically.  To do this we need to ensure that the outgoing message contains our ‘'special reply text string”. We can accomplish this by the use of a ‘hidden’ HTML tag at the start of the outgoing message as follows:

#-#- Text following text is ignored –#-#

 [The ‘#-#-‘ string (and its derived inverse) is a component setting and can be any desired unique string that is unlikely to be provided by the user in the usual course of business.]

We add this to the email template body and it has the advantage that it will not be displayed in the message due to the span style which hides the text. If the user replies and includes the received message contents in their reply it would be picked up by the message body checking code and can then be eliminated from the message stored in the component issue.

This will not of course eliminate all unnecessary text from the ‘email update message’ but it can go a long way towards reducing it. 

Making a component truly multilingual (5)

Whilst finishing up our component there were a few matters which we needed to address.

 

Site Map Generation

The first of these related to the generated ‘site map’ which was not displaying correctly. The problem was caused by us omitting to include the ‘new’ alternative language main menu items in our site map generation component (OSMAP/XMAP) which soon resolved the matter. To be more precise the ‘site generator component’ was installed prior to our multilingual work upon the site, so did not ‘automatically’ pick up the menu items it uses in the map generation. Obvious once one looked at it. It did reveal an addition place where we had to add out additional ‘'database query condition to include only the current displayed language (plus those marked all or undefined).   This just required adding to the ‘getCategories’ method, although to be absolutely certain that only the relevant entries were shown, we also added it to the ‘GetEntries’ method as well. This latter is probably not required except where entries are added to the incorrect categories, but acts as an extra check when the entries are generated.

Deletion of Associated Items

This actually applies to both the front end and the back end of the site. We decided to take a deeper look at the code to handle the deletion of any associations when an item is deleted. To be more precise when an item is deleted, the existing association entry for that item should also be deleted from the associations table. As originally written all association records would be removed for the deleted item and hence also for the associated items. However this is not necessary correct when there are three (3) or more associated items, since one really only wished to remove one association record.  For two associated items it is fine, since we also wish to remove the ‘other’ record in the associations table at the same time. .  So to be precise, we should for situations where there are more than 2 associations for an item, only remove the single record in the associations table, but if there are only two item, then we should remove both records from the associations table. This has worked well in all our tests and wonder why some similar code does not exists in the base Joomla components such as com_content etc. On a busy site with items being created and deleted on a regular basis we can see the situation where the database associations table would be much larger than is necessary, since there would be without similar code being present in the components, resulting in a slower response to other queries.

 

How the PHP tests work

One of the things we omitted to mention in our last post was how the actual code checks were working and didn’t explain what the checks were actually ‘checking’. We made use of two specific methods which we expand upon below:

JLanguageMultilang::isEnabled()

This method determines if the language filter plugin is enabled.  It works for both site and administrator on the site.  If the site is supporting multiple languages then true is return else false is returned.  The presence of the language filter plugin is what is determining whether multiple languages are being used upon the site. Note it does not check whether multiple languages are actually installed.  The distinction is subtle but very clear.

JLanguageAssociations::isEnabled ()

This method is used to determine if the language filter Items Associations parameter is enabled.  One can always have a multilingual site and not make use of item associations, which is ‘relating’ items that are ‘effectively’ the same, but only expressed in a different language. An example might be a site located in a country with two different locally spoken languages (i.e. Serbia) and items are translated and presented in both languages to site visitors.

Making use of the two standard Joomla provided methods simplified our coding, however we still had to add our own code to pick up any desired associations for our displays.

 

To be continued…

Go To Top

Joomla! Debug Console

Session

Profile Information

Memory Usage

Database Queries