Macrotone Blogs

Macrotone blogs upon Joomla, our products and other matters.

An IT professional with a wide experience of IT systems, specialising in Database Management and Security.

Using a custom Joomla library in your components

We were aware of an article written by Roberto Segura which covers the use of libraries in ones; own written components.  This addresses (in one way) a conclusion that there was a certain amount of duplication particularly in the language strings but also in code that is used both in the front end of a Joomla site and in the back end of the site.

We have been ‘playing’ (for want of another word) with the concept in a new component that we are developing and are very pleased with the way that it is shaping up.  We had already made changes in some of our later components to use the back end language strings in preference to the front end language files, and also where ever possible use the Joomla globally defined strings, but the use of a specific component library can aid in the use of language strings but also in the use of common code, without the need for adding additional code to include specific files by making use of the Joomla library autoloader.

We thought it was worth sharing this with our readers as even though we have been using Joomla for many years there is still much more to learn and discover. This is after all another one of the things we like about Joomla.

Message from local network scanner – jpaoscanner

An empty or blank email with the subject of Message from local network scanner pretending to come from jpaoscanner at your own email domain  with a malicious word docattachment  is another one from the current bot runs which try to download various Trojans and password stealers especially banking Trojans like Dridex or Dyreza and ransomware like cryptolocker or Teslacrypt.

The attachment to these are named Scann16011310150.doc   Evidently some attachments have a letter F after the doc which effectively makes them useless because windows doesn’t know what to do with them and asks you. They will open in Word, if you tell them to, and do contain a malicious macro that will infect you  

We have started seeing these messages and searching the web reveals we are not alone.

See these links:   myonlinesecurity and Dynamoo,  amongst others    

Goodbye 2015, Hello 2016

roadmap

The year 2015 is now gone and it was time to look at how well we met our 2015 Roadmap

We set out 5 specific items that we wished to address as follows:

  • New component release (or two) based upon in-house Joomla developments.
    • 100% We think we achieved this by releasing both Rialto: our Certified Advertisement component and Kbase: our Knowledge base component
  • On-going development of components adding new features/improvements as identified.
    • 100% We continued to release updates of our components throughout the year, so consider this a success as well..
  • Content & layout on Macrotone Consulting  website needs review, fix typos, better layout, etc.
    • 100% Using our Kbase component we restructured our web site articles to use the new component. That is not to say there is not more that could have been achieved but overall it was successful.
  • Update our JED entries reflecting the new format.
    • 0% This one we still have to achieve, so we will carry this one over to the new year. 
  • Remove any component releases, and code in existing components not specifically for Joomla 3, as Joomla 2x is no longer supported.
    • 75% Partial as a number of our customers continue to use Joomla 2.5 which made it difficult to remove ‘old’ code. However most have been updated and the remainder will be completed in the next few months.  Joomla 2.5 is of course 12 months out of support so it is not unreasonable to not be able to support it any longer.

So we hope to achieve the following in 2016

  • New component release based upon current in-house Joomla developments.
  • On-going development of components adding new features/improvements as identified.
  • Update our JED entries reflecting the new format.
  • Complete removal of component releases, and code in existing components not specifically for Joomla 3, as Joomla 2x is no longer supported.
  • Joomla 3.5 supported component versions.

Password Control User Profile Plugin 0.0.5 released

We are pleased to announce an updated version 0.0.5 of the Password Control User Profile Plug-in.

This optional plug-in works with the Password Control System Plug-in to display information in the users profile of their last and next scheduled password change.  It has been updated to also display the 'forced change flag' which was introduced in the System Plugin 0.1.5 release to prevent a bypass of the password change request.  It works with version 0.1.5 and up of the system plug-in.

The user plug-in also provides the site administrator with the ability to change a user's next password change date, subject to the settings specified on the system plug-in parameters.  See the documentation for more details.

Joomla 3.4.x and Mootools

joomlaWe have long been aware that the use of Mootools in Joomla extensions is being discouraged, but also know that it is used in a number of different places in the core code and would be automatically loaded if required.

What we were not expecting was that relatively new extensions would be using it.  We have just spent the best part of a week playing with a particular problem involving Mootools that we thought we would share.  We tended to make things more difficult for ourselves as we were using a plugin to automatically disable Mootools (for a much older problem), which tended to ‘mask’ possible solutions which meant we would have reached a conclusion much earlier if we had remembered.

We were working on an update to a front end form and observed that calendar fields were not displaying the ‘popup’ calendar picker window, also that when displaying the ‘tags’ field to enable the user to select or create new tags to add to an item, that the displayed field did not match what we were expecting, or seeing in the back end of the sight using the same code constructs.   We could see a Javascript error in the error console that was either a) Tips not found, or b) Squeezebox not found.  The first coming from a form field tool tip, and the second coming from the form calendar field.  These were obviously related to the form elements, but the tag display didn’t appear to have either of these used in its construction.

The tag display with Mootools disabled looks as follows:

20151221154101 image 4

We tried adding in explicit call to load various Mootools elements yet were not seeing any change in the problems.  After much thought and investigation it was then that we realised that we had the Mootools disabler plugin enabled so all our changes were effectively being ‘ignored’.  After a little silent ‘blue’ language we disabled the plugin and the Javascript errors disappeared and the calendar popup was visible again. 

What we had not expected was that the Tag display would also correct itself, which effectively meant that the Joomla tags helper must be using Mootools somewhere in its code. 

The desired tag display form field required (and achieved):

image

Despite searching we couldn’t find any reference that documented this, so its one to watch out for if you meet it yourselves.

Rialto v1.2.0 released

rialtoWe are pleased to announce release 1.2.0 of the Classified Ads component for Joomla named Rialto.

This component runs on Joomla 3.4 (and above) only. 

This release adds two major features, the first is support for Joomla Tags.  The second is that the front end Entry edit now makes use of JForms for better and easier control. One other significant change is the implementation of Joomla ACL controls, although this release does not fully implement it in the front end of the site. This will be more fully used in future releases. There are a number of minor changes/fixes also implemented in the release.

See the change log and documentation for more details.

The release is available in the download area.

Tracking down CSS icon problems

We recently noticed a strange effect upon our web site, when we implemented a new component which was making use of the Joomla Searchtools and the JHtml 'jgrid.published' method upon the front end.  The basic problem was that the methods which use various icons such as icon-search, icon-publish icon-unpublish etc. was not displaying the icons and instead all we were observing was a small square rectangle where the icon should have been shown.

It was apparent that it had to be a site template ‘feature’ since when using the Joomla Protostar template all icons were correctly displayed.

We have made use of a Bootstrap 3 based template for some time and not observed this effect before, which is probably because we had not tried to use these Joomla methods before in the site front end.

Using the Firefox Web Developer plugin, we could see the CSS that was generated, and we could try and edit the CSS until we could achieve  the desired result, which was obviously to see the icons. What puzzled us for a little while was why after adding what we thought were the correct CSS commands the screen display (Firefox) didn’t immediately show the icon specified, and instead displayed a small lined box icon.  This made us immediately think that we had the wrong code, but after adding the same codes to the actual template and refreshing the page, clearing the cache etc., the requested icons did indeed appear.  Whether this is a Web Developer feature is unclear but it was certainly misleading!

The Bootstrap 3 template makes use of Awesome Fonts and it appears that something or somehow the font family has been disassociated from the icon class, so we added a few additional CSS rules to the template and this resolved the problem.

.icon-publish:before, .icon-save:before, .icon-ok:before, .icon-checkmark:before {
font-family: FontAwesome; content: "\f00c";
}
.icon-unpublish:before, .icon-cancel:before {
font-family: FontAwesome; content: "\f00d";
}
.icon-trash:before { font-family: FontAwesome; content: "\f1f8"; }
.icon-search:before { font-family: FontAwesome;    content: "\f002"; }
.icon-archive:before { font-family: FontAwesome;    content: "\f187"; }

The above FontAwesome content codes are for the 4.4.0 release.

Whilst investigating this we also looked at a problem that we has observed some time ago but never resolved, which was that the small drop down that we present on some pages which contains the icons to Print or Email the page contents, was difficult to see, and only upon hovering with the mouse over the small box where the dropdown was positioned, actually revealed its presence.  We wished the presence of the box to be more visually evident.

Again Firefox Web Development Tools helped and we could see that the dropdown CSS code was effectively empty so we had to specify a background colour as below:

a.btn.dropdown-toggle
{
background-color: #99ccff; 
}

This added a nice light blue colour to our background and achieved what we desired.

We are still a little puzzled as to why the above was required, but suspect that the reason is that Joomla itself has implemented specific changes to its own Bootstrap 2 implementation as used by the Protostar template, as since we were/are using a template based upon Bootstrap 3 we need to add these additional CSS codes to ensure functionality is maintained. This tends to support our conclusion from an earlier problem with the use of vertical tabs, which are not supported in Bootstrap 3, but that is a different post topic completely!

Tags:

Perseverance wins the day!

We recently investigated a problem upon our web site that impacted our Forum, and the ability to submit new topics or edit 'old' topics.
The problem was complicated in that there were no error or warning messages to provide any indications of what may be the cause. The most obvious symptom was that all of the problems were related and concerned the inability to submit 'form' data. i.e. The login process submits the entered details to the login process.  The creation or editing of a forum post involves submitting 'entered data' to the post process.  

Our immediate thought was that we were seeing a jQuery conflict, since we have seen similar problems in the past, again with no indication of the cause. We then proceeded to look at all the components, modules and plugins that we use upon the site but despite intensive searching we couldn't find anything that might be the cause, and there was certainly only one version of jQuery being loaded.  Investigation also revealed that only forms submitted by the forum software were impacted.  Very strange and seemed to indicate the forum software itself was at fault, but nobody had reported such a problem and when we tried it on another web site it worked perfectly.  We also tried installing an 'older' version of the forum software and this didn't resolve the problem either.

One complication was that we were also having problems with our backup software, and found that it was not possible to restore a backup to enable to perform some tests upon a copy of the live site.  We tried the software upon an earlier version of the site and it was working perfectly. This just made the problem more difficult to understand since that particular site was more likely to have a problem since it was more complex.

Once we have a solution to enable us to restore a current backup working correctly we performed a restore.  This was interesting since the 'restore site' worked perfectly and didn't exhibit the problems seen on the 'live site'.  This meant that the problem had to be caused by either a) the operating software upon the two platforms, b) the version of the underlying database, or c) the version of the PHP software used by the site.  Looking at the technical requirements of the software all were being met but there had to be something getting in the way.

Upon a hosted platform there is very little if anything one can do about the operating system, so we were left with the PHP and/or database versions. We looked at the PHP versions and tried a number of versions including 5.4.44, 5.5.28 and 5.6.12, without any change in the problem. Changing the database version is more complex since it required the creation of a new database and a restore.  The database versions upon both system was virtually the same being MySQL upon the live site and MariaDb upon the test site, although they were the same versions.

We therefore took the decision to try a system restore upon the live site with very little hope of a solution. To our surprise a system restore corrected the problem, which leads one to suspect that there was some corruption in the file system somewhere, but we currently have no idea exactly where at the moment. The most likely is that one or more of the forum software files was corrupt.

This was a long and exhausting investigation covering many days and resulting in delay to other activities.  Looking back if our backups had been working correctly then we might have got there sooner, which makes one ask just how regularly does one test the backups.  Obviously not enough has to be the answer in our case, even though we do test them periodically.  I suspect we are not alone in not testing the backups sufficiently often.

Tags:

MariaDB and Joomla update

Further to our original post, we have now carried out a series of tests and are pleased to say that we have found no problems with using MariaDB with Joomla.  Our tests compared version 5.5_44 of Maria DB and MySQL and to all intents and purposes no difference is noticeable.  We cannot say there is any difference performance wise, since we were testing upon different server instances, which makes such a comparison impossible to determine.

We can also say that our NAS in now running MariaDB as standard, so we will be using this version (5.5.44) for all our development work henceforth.

“For all practical purposes, MariaDB is a binary drop in replacement of the same MySQL version (for example MySQL 5.1 -> MariaDB 5.1, MariaDB 5.2 & MariaDB 5.3 are compatible. MySQL 5.5 is compatible with MariaDB 5.5). What this means is that:

  • Data and table definition files (.frm) files are binary compatible.
  • All client APIs, protocols and structs are identical.
  • All filenames, binaries, paths, ports, sockets, and etc... should be the same.
  • All MySQL connectors (PHP, Perl, Python, Java, .NET, MyODBC, Ruby, MySQL C connector etc) work unchanged with MariaDB.
  • The mysql-client package also works with MariaDB server.
  • The shared client library is binary compatible with MySQL's client library. “
Tags:

Joomla Smart Search Index size observations.

We have recently upgraded the underlying database version of our Joomla site to MySQL 5.5, from 5.1 and carefully observed the sizes of the underlying databases as we progressed. The source database was reported as being some 200Mb in size, which is not excessive for our modest site, but we were very surprised when after restoring the system to the new database the database size was reported as 37Mb.  We understand how databases store their data, so it was not the fact that it was smaller than was the surprise more the dramatic difference.

We then proceeded to rebuild the ‘Smart Search (Finder)’ indexes and checked the database size yet again.  This is necessary, since the backup does not by default include the Smart Search table data in the dump file. It was now reported as being 147Mb, and increase of 110Mb.

The fact that this increase was on our modest site, must make one wonder whether the benefits out weight the database storage overhead.  Ours’ is certainly not a large site, and the use of ‘smart search’ on large sites is not something we would wish to consider, and a dedicated standalone search engine, such as Solr, or possibly the Google Site Search, rather than the pure-PHP implementation used in Smart Search may well be found to be more suitable.

Smart Search itself consists of a number of ‘tables’ in the Joomla installation, of which at least 16  are "mapping tables". These mapping tables resolve the many-to-many relationship between search terms and the content items that contain them. In theory those 16 tables could be merged into a single table and doing so would have negligible impact on small sites. However, on larger sites there would be a substantial impact on performance because the mapping table would have a very large number of entries and updating such a large table can be time-consuming.  Maintaining the search term index requires a fair amount of disk space, which increases the more terms that content items contain. Since phrases are also indexed, the longer the phrases, the more disk space required. For Smart Search a reasonable compromise has to be reached between search usability and disk space usage.

Our observations must question this compromise and possibly we need to review the settings that we are actually using. It is likely that most readers may not be aware of exactly how large the ‘smart search tables’’ are in their own Joomla installations.  The decision as to whether to continue using the Smart Search’ component should also be reviewed.

Go To Top

Joomla! Debug Console

Session

Profile Information

Memory Usage

Database Queries