Macrotone Blogs

Macrotone blogs upon Joomla, our products and other matters.

An IT professional with a wide experience of IT systems, specialising in Database Management and Security.

MySQL 1071 and associated errors.

mysqlOur users have reported a few MySQL errors over the years but one of the most common appears to be the

ERROR 1071 (42000): Specified key was too long; max key length is xxx bytes

Where xxx is some specified value often, but not always, a value of 767.

For that reason we decided that we would make this blog entry to try and explain the situation.

We are concerned here mainly with InnoDB tables since that is what we tend to use in our extensions.

Continue reading

Newsfeeds display – solution to an enigma

b2ap3 icon joomlaWe have always displayed a few relevant newsfeeds upon our web site, but it has never, to be honest been very high on the visitor list, or upon our own priority list. We long ago noticed that the default display for the ‘Newsfeeds Categories’ in the front end of our site comprised merely of two lines, one for each of the two newsfeed categories we use, which also acted as a link to the underlying newsfeeds in the respective category.  There was no page header, no display of the breadcrumb information, and no details of the category descriptions.  In short  a very barren page.

Originally it was suspected that it might be a template or CSS type problem.  Attempts to change the menu settings, resaving module specifications etc.,  all proved fruitless.  It didn’t matter what the menu settings were, they were silently ignored.  One is tempted to say it was a cache type problem, but bearing in mind that this has been the situation for several months, if not longer  and the various caches’ had been manually cleared several times during that period, it was obvious that something else was amiss.

With the change to a new site template, the situation remained unresolved and it was starting to get a little bit annoying.   So given a hour or so spare we decided to investigate further.  Inspection of the PHP code underlying the display revealed no clues, and despite retrying all our previous steps we were no further forward.

Searching the web for similar reported problems drew a complete blank, apart from a link to a very strange problem we had ourselves encountered with the display of breadcrumbs on a previous occasion.  Looking back we found out previous blog entry. [So keeping a blog can prove useful.]  We thus decided to clear all URL entries from the sh404SEF component for the com_newsfeeds component.  Low and behold on refreshing our web page the correctly formatted page was shown, complete with headers, breadcrumbs, descriptive text etc.

We realise that sh404SEF keeps track of URL links, but why this should impact the page display is currently a bit of a mystery.  It doesn’t itself cache pages, but must somehow also keep track of which modules and what the ‘previous’ settings were for a page ,for which it is keeping a record of the link.  I am sure that I have never read anything of this sort in the component documentation.

What we learnt gain from this is that sh404SEF seems to have some strange characteristics which impact what is displayed upon a screen, far and above just converting non SEF URLs to a SEF format.  So it you are ever seeing a similar type of problem and every thing else seems to failing to resolve it, it might, if your site is using sh404SEF be worth clearing your entries and seeing if it resolves the problem.  Certainly stranger things have happened.

Canvas Fingerprint code tracking

fingerprintThe topic of the moment appears to be ‘Canvas Fingerprinting’ with a number of articles available on the web. It is the latest development in use for tracking the movement of users on the web. You do not need to click on a widget to be tracked, just visiting the site is sufficient.  It exploits the subtle differences in the rendering of the same text to extract a consistent fingerprint that can easily be obtained in a fraction of a second without the user being made aware.

A research paper concluded that code used for canvas fingerprinting had been in use earlier this year on 5,000 or so popular websites, unknown to most of them. Most but not all the sites observed made use of a content-sharing widget from the company AddThis.

The mechanism: Canvas Fingerprinting works in a similar way to cookies, by keeping a record of which sites are visited. When a browser loaded the AddThis widget, JavaScript that enabled canvas fingerprinting was sent. The script used a capability in modern Web browsers called the canvas API that allows access to the computer’s graphics chip, which is intended for use with games or other interactive content.

An invisible image is sent to the browser, which renders it and sends data back to the server. That data can then be used to create a “fingerprint” of the computer, which could be useful for identifying the computer and serving targeted advertisements.

But of several emerging tracking methods, canvas fingerprinting isn’t the greatest: it’s not terribly accurate, and can be blocked.  The Electronic Frontier Foundation (EFF) recommend their own ‘Privacy Badger’ or the Disconnect add-on.  

The list of sites that still track you is at this address.

So much for privacy.

ReCaptcha disappeared from User Registration.

ReCaptchaWe were recently informed of a problem on our site where the ReCaptcha image was not being displayed.  This was puzzling since we hadn’t changed anything and the image was showing in other locations upon the site, such as adding article comments etc.

We were aware of a few changes a year or so ago by Google but are not aware of any recent changes.  After some searching we found this article which described our problem and provided a solution that appears to work well.

We had already checked that Method 1 in the article didn’t resolve the problem so we used Method 2 which we reproduce below.

This overrides the Core Joomla code for ReCaptcha:
1. Open the file: /plugins/captcha/recaptcha/recaptcha.php
2. Find the following string in the OnDisplay function:

return '<div id="dynamic_recaptcha_1"></div>'; 

3. and replace with:

   1:  // Replace YOUR_KEY with your public key 
   2:   
   3:   return '<div id="dynamic_recaptcha_1"> 
   4:   <script type="text/javascript" src="http://www.google.com/recaptcha/api/challenge?k=YOUR_KEY"></script> 
   5:   <noscript><iframe src="http://www.google.com/recaptcha/api/noscript?k=YOUR_KEY" height="300" width="500" frameborder="0"></iframe><br> 
   6:   <textarea name="recaptcha_challenge_field" rows="3" cols="40"></textarea> 
   7:   <input type="hidden" name="recaptcha_response_field" value="manual_challenge"></noscript> 
   8:   <script type="text/javascript"> 
   9:   window.onload = function() { 
  10:   Recaptcha.focus_response_field(); 
  11:   } 
  12:   </script> 
  13:   </div>'; 

 

 

We have tested using Firefox, Chrome and IE and in all cases we now see the ReCaptcha image, where as previously we were not.

The following additional comments were present in the original article and also apply in our situation.

Additional

  • This applies to a case where the code WAS working and without any changes to the server (no updates), and the recaptcha simply stopped displaying.
  • Public Key and Private Key have been setup in the Joomla Core Plugin.
  • Set to display in both User Registration and Global Configuration.
  • K2 is NOT installed.

Note Well:

This change would have to be reapplied if the plugin is updated by any future update from Joomla, but at least the site user registration is working correctly!

Pavlovian approach to Password Management.

Engineers as Stanford recently unveiled a new password policy that shuns one-size-fits-all security.  This has been followed a system proposed by Lance James, the head of the cyber intelligence group at Deloitte & Touche.   This proposes a system that provides rewards or penalties based on the passcode choices people have made.

The example given is one as follows:

A user who picks "test123@#" might be required to change the password in three days under the system,. The three-day limit being based upon calculations showing it would take about 4.5 days to find the password using offline cracking techniques. Had the same user chosen "t3st123@##$x" (all passwords in this post don't include the beginning and ending quotation marks), the system wouldn't require a change for three months.

An interesting concept, and on that would avoid forcing users who have made a sensible password choice from being forced to change their passwords because some other less careful users choose ‘easier passwords’.

The full article is here.

Privacy Badger– an interesting Browser plugin

There is an interesting plugin for Chrome and Firefox currently in ‘Alpha’ release from the EFF (Electronic Frontier Foundation) who brought us ‘HTTPS Everywhere‘ named ‘Privacy Badger’.

Privacy Badger is described as a browser add-on that stops advertisers and other third-party trackers from secretly tracking where you go and what pages you look at on the web.  If an advertiser seems to be tracking you across multiple websites without your permission, Privacy Badger automatically blocks that advertiser from loading any more content in your browser.  To the advertiser, it's like you suddenly disappeared.

More details upon the EFF website.

Web access URL’s containing ‘RK=0/RS=’ string

We have noticed over the past few months an increase in the number of web access upon various URL addresses upon our site with a string starting ‘/RK=0/RS=’, followed by strings of other characters.  To us they are obviously some attempt to get access to information but we were a little puzzled as to how they might possibly work. The URL’s they are attached to are varied but seem to be upon a lot of Blog addresses. The RS= looks like it could be a regular expression for a pattern match of sorts, since some(but not all) are sometimes followed by a caret ^ but that is speculative.

They look to be a form of  SSI injection with the header, with the attempt to try and pass tokens into the URL for some purpose..

Apparently we are not alone and there is much discussion upon the web as to exactly what it is trying to achieve and who might be behind it, but no clear answer is currently known.

One way to remove them might be a simple .htaccess rule similar to the following:

RewriteRule ^(.*)RK=0/RS= /$1 [L,NC,R=301]

An alternative would be to block the IP addresses from which they are coming, but if they are not ‘hard addresses’ in the sense that they are not reusable,  then the risk is that you may end up blocking legitimate traffic.

Longaccess: pass your digital assets to your heirs

longaccessOne question that is often asked is how one preserved ones’ digital assets and pass them on to your heirs.  We recently read about a new service that may offer a solution.

Longaccess promises to be a cold storage of sorts for your digital life. It's a cloud-based service that operates off Amazon's S3 data centres, but unlike other file lockers such as Dropbox or Google Drive, Longaccess aims to be less accessible, but more dependable. It describes itself as a ‘safe’ on the Internet, a location where one can store files fully encrypted and secured, safe and ready to be accessed for decades.

Longaccess is not a file syncing service, nor is it  a file sharing service.  It is a service for storing files for long periods of time. Files that are NOT updated, or changed at all. Every time a file is created and uploaded to a Longaccess Archive using the desktop application, one gets an Archive Certificate.  This is a simple text file, that contains all the information required to access the data in the future:

- Anyone with access to the Archive Certificate can access the corresponding Archive data: Nothing else is required, not even a username or password.

- Access to the Archive data is impossible without the corresponding Archive Certificate. No one, not even the owner, nor Longaccess, can decrypt the Archive without the Archive Certificate.

One can think of the Archive Certificate as a full entitlement to access the data of a specific Archive. If one gives a copy to someone else, they can also access the data.

There are a number of questions re cost etc. that immediately spring to mind, including how they can guarantee they will be around in a decade or so, question which they try to answer on their web site.

Sounds interesting and may well be a way to preserve those ‘old’ photographs for posterity.  One that may well be worth watching for a future opportunity.

Test email in foreign languages

We have been working with the emailing of problem reports to our Issue Tracker component recently in particular with the specific problem of languages using ISO-8859-2 character sets.

Having made some code changes to handle this, we were thinking of how it might be possible to test out other languages, with other character sets such as Chinese, Korean etc.

The little grey cells starting thinking about the various translation sites upon the web, and whether there might be any that could not only perform a translation of some specific text but also complete the task by emailing the translation to a specific address.

After some searching it seems that this is not an unusual requirement and we found several that could possibly do what we required. A lot only handled the translation part of the requirement, but the sending of the email was not that common. It was important that the email was sent from the third party since if we used a local email client the details in the message header and body did not accurately reflect the correct character set in use, and this was the one thing we wanted to test.

A number of sites imposed limitations such as the number of characters in the message, the number of messages that could be sent etc., which is generally reasonable since they are endeavouring to make a living from providing a service and would prefer to charge.  However these limitations were not of a major concern to us, especially as the text content could be anything at all, as long as the character sets were represented.

We obviously will not list all of the sites we investigated but the one which we found suitable for our needs was WorldLingo and though it insisted in creating accounts for both our sender and receiver of the generated emails, this was something we could easily live with. There is a vast range of possible languages to choose from, certainly more than we will ever use or test I suspect, and the machine translations were more than adequate for our purposes.

Our requirements were not all that unusual at all, and I suspect others might have the same sort of need, in which case hopefully this may act as a pointer.

Update: One interesting side effect we noticed was that, when we sent the email (via WorldLingo) it was 'processed' by our component and automatically send a reply acknowleging receipt. Since the emails from WorldLingo are all sent out with individual identifers in the email address the reply was sent to the named worldlingo address which then forwarded it to use (the sender). The interesting aspect was that the text was 'translated' on the reply and didn't quite match what was the 'original' text in English. One of the interesting aspects of translating in this case from English->Japanese->English. Not a concern to use but just goes to show how things can get confusing in translation.

Issue Tracker Template Overrides

b2ap3 icon joomlaWe have recently been ‘playing’ with a new ‘Bootstrap v3’ template for the front end of our site.  This involved use creating a set of template overrides for our Issue Tracker component and we decided to share the details with our users.

Joomla has long had the ability to create Template Overrides, which are modifications to the Joomla components or modules. This permits changes to be made upon a ‘local site’ basis without the need to change or hack the supplied code.

We are primarily concerned with the Issue Tracker component and we have tried hard to produce front end displays of Individual Issues and of the Issue Entry form that would be usable in the majority of installations. However the differences between the various template used on sites are many and vast, and it is almost inevitable that they will not be suitable for everyone. This was indeed the situation we discovered ourselves when using a BootStrap template for the site.

Continue reading
Go To Top

Joomla! Debug Console

Session

Profile Information

Memory Usage

Database Queries