Macrotone Blogs

Macrotone blogs upon Joomla, our products and other matters.
Blogs that do not fit into any other of our specified categories.

Broadband Connections

We have recently relocated and this in itself whilst being ‘fun’ has in particular provided a number of ‘opportunities’ as regards our broadband access mechanism.

Having been a BT customer for  many years our first thought was to continue using them as a broadband provider. Alas this was not possible since our ‘landlord’ didn’t want lines ‘over the property.   OK you might say, but just cancelling BT broadband was itself ‘interesting’ in that they see this as an opportunity to make money.  Not only is there a disconnection fee, but also since we paid in advance for line rental for a year they deem that any of the rental paid for the remaining part of the year is also lost. This latter cost can in itself amount to over £100 pounds.  So make a note to never pay in advance for BT services since if there is ever a need to cancel them then it is going to cost you more than you expect.

The intent was to instead make use of a cable connection.  Cable was laid to the property and all that was required was for the cable providers to come an make a connection. Sounds simple, but no, even here the cable providers fail to turn up and carry out the work, due to ‘a driver being ill’.   Strange that they only have one driver you might think, and you would not be alone. Here we are several weeks later and we are still waiting.

So we have ben forced to look at other alternatives. Many years ago we made use of a ‘wireless mobile broadband’ connection from BT.  This is effectively a dial up connection using a mobile instead of a modem.  It worked quite well but was very slow.  Dusting off the device and trying it didn’t work, mainly we suspect because if one does use a ‘mobile’ number in a period of time, often about six months the mobile number is effectively dead.   Not too surprised at this but it got us thinking and sure enough things have moved on.  Seems the current solution is ‘wireless hub’ which is a small device about a few inches square, into which one insets a mobile SIM card and once connected provides internet facilities for up to 10 devices.  These seem to be provided by most if not all of the mobile providers, and one contracts for a ‘data’ only SIM device.

There are a few things to consider.  First what is the likely usage of the data, and in particular how much data are you likely to use each month.  Often one is totally unsure of how much data would be viewed in a month, but one has to have a rough estimate, since if one exceeds the ‘monthly’ contracted usage, the use is either a) cut off for the rest of the month, or b) subject to an increased cost for the data used after the limit is exceeded.  Neither of which is a good outcome.

One contracts for a certain amount of data usage each month, and different providers have different set levels, such as 500Mb, 1Gb, 2Gb, 4Gb, 8Gb etc.   The cost is also somewhat variable between different providers.  An additional consideration is how satisfied the customers of each mobile provide are, and this alone is sufficient to eliminate a few of the possible suppliers.

Remembering our previous performance observation we were really only interested in using a 4G network. Those of you still using a portable phone on a 3G network will be aware of how slow surfing the web can be.  We selected a 30 day contract with EE for a 4G mobile SIM and so far we have been very happy with the result.  Performance wise it is faster then the fixed line speed we were used to seeing using our ‘old’ BT connection.  Cost wise, it is a little early to say, but since we are not paying for a ‘broadband connection on a fixed line’ this tends to offset tie cost a little, hence we are not really seeing a real difference per month.

Whether this is a solution for the long term, remains to be seen, but as a short term solution is is working well and should not be discounted if one ever sees oneself in a similar situation in the future. 

CDN hosted sites, Tor browser and Captcha

We were recently making a modification to our IP Mapping component to support CDN sites such as Cloudflare as a result of a recent forum post, and we discovered the answer to a observation that we had seen a few times that we thought worth sharing.

We occasionally use the Tor browser to access web sites, usually to give us a random set of IP addresses that we can test upon a site, when using IP Mapping.  It is a very convenient way in which one can test access to a site, and appear to be coming from somewhere else in the world. We had observed that occasionally we were presented with a captcha page on some sites as shown below:

 20160502093024 Cap1 2

In the example shown we are accessing the Cloudflare site itself.

We hadn't worked out why this was occurring but now believe that it is something that sites hosted by Cloudflare sometimes display.  We think this is Cloudflare itself that is intercepting the IP address that the Tor browser is using, i.e. the specific Onion exit IP point, and that Cloudflare is then deciding to display the captcha.  If is probably not that difficult to do, and only requires a mechanism to keep track of all the possible Tor access points, and if the browser is coming from one of these IP locations present the captcha challenge.

Of course this makes some sense since Cloudflare is presumably protecting the sites it is hosting, but to a visitor (using the Tor browser) it is not evident or always known that Cloudflare is hosting the site, so it may come as somewhat of a surprise.

Of course other CDN sites may also be using such a mechanism as well so if you see such a captcha mechanism in place it may not be the site you are accessing that is the source of the captcha but the CDN site itself.

We have only observed this behaviour when using the Tor browser, and note that Cloudflare has a mechanism to let the hosted site decide what action to take when the Tor browser is used.  Other CDN based sites and other browsers may exhibit similar ‘opportunities’ but of these we are not (yet) aware.

Message from local network scanner – jpaoscanner

An empty or blank email with the subject of Message from local network scanner pretending to come from jpaoscanner at your own email domain  with a malicious word docattachment  is another one from the current bot runs which try to download various Trojans and password stealers especially banking Trojans like Dridex or Dyreza and ransomware like cryptolocker or Teslacrypt.

The attachment to these are named Scann16011310150.doc   Evidently some attachments have a letter F after the doc which effectively makes them useless because windows doesn’t know what to do with them and asks you. They will open in Word, if you tell them to, and do contain a malicious macro that will infect you  

We have started seeing these messages and searching the web reveals we are not alone.

See these links:   myonlinesecurity and Dynamoo,  amongst others    

MariaDB and Joomla update

Further to our original post, we have now carried out a series of tests and are pleased to say that we have found no problems with using MariaDB with Joomla.  Our tests compared version 5.5_44 of Maria DB and MySQL and to all intents and purposes no difference is noticeable.  We cannot say there is any difference performance wise, since we were testing upon different server instances, which makes such a comparison impossible to determine.

We can also say that our NAS in now running MariaDB as standard, so we will be using this version (5.5.44) for all our development work henceforth.

“For all practical purposes, MariaDB is a binary drop in replacement of the same MySQL version (for example MySQL 5.1 -> MariaDB 5.1, MariaDB 5.2 & MariaDB 5.3 are compatible. MySQL 5.5 is compatible with MariaDB 5.5). What this means is that:

  • Data and table definition files (.frm) files are binary compatible.
  • All client APIs, protocols and structs are identical.
  • All filenames, binaries, paths, ports, sockets, and etc... should be the same.
  • All MySQL connectors (PHP, Perl, Python, Java, .NET, MyODBC, Ruby, MySQL C connector etc) work unchanged with MariaDB.
  • The mysql-client package also works with MariaDB server.
  • The shared client library is binary compatible with MySQL's client library. “
Tags:

Issue Tracker: Handling received emailed issue updates

issues-48Our Issue Tracker component for Joomla has long had the ability (when configured) to handle ‘issues’ sent to a predetermined email address upon a mail server, and to automatically create issues. What is not quite so well known is its ability to handle updates to existing issues via received emails as well.

Certain criteria have to be established to ensure that the ‘update email’ relates to an existing reported issue.  There are several steps to this.

a) When an issue creation notification is send out the sent email contains a couple of ‘special’ mail header tags.  When an update email is received the presence (or absence) of these mail header tags are checked, and provide one level of confirmation that the ‘email update’ is for an existing issue.

b) The email title is also checked for the existence of a special format string.  The format of this string is usually [Issue: xxxxxxxxxx] or just [xxxxxxxxxx], where xxxxxxxxxx is the issue number. It is possible to have different formats for the issue number which may be a ‘random’ ten character string, or a nine digit number preceded by zeros and a leading character (again ten characters in total), or a numeric string consisting of up to 10 digits.

[Release 1.6.8 of Issue Tracker enables a slightly more relaxed format string in that any character string may precede the issue number. The square braces are required in all cases.]

We can assist in the issue number detection by specifying one of the above issue format strings in the outgoing message notification. This is a component configuration and the format of message header and message body contents is possible for all outgoing message types.

c) The email senders address is also checked to see whether it matches the email address of the person who first raised the issue report.

If the criteria is not met then the email update is rejected.  If however the email is identified as being an update for an existing issue, then the handling of the email body has to be looked at.  A number of email clients (all?) such as Outlook, Thunderbird etc. (to name just a couple) permit ‘reply’ emails to incorporate the received email text in the outgoing message either preceding or following the ‘new’ message text. This is potentially annoying in that if the body text was taken as it is received, the message would contain unrequired information, which is already present in the issue ‘record’. We therefore seek to eliminate the ‘former unrequired included text’ from the message before saving the data in the issue record.

The easiest way to be specify a special ‘text string’ which is searched for in the message body and any text seen after this ‘string’ is silently ignored.  This works well in practise but only if the ‘string’ is present in the message.  If it is not present one is left with a decision as to whether to accept the whole message, ‘included former messages text’ or just reject the message entirely.  This is obviously a decision that a configuration option permits a site to make.

It is not possible to know whether the client browser, is going to include the ‘former message text’ either before or after the ‘new’ message content,and there is no standard way for a client to indicate the present of this ‘former’ text in the message.  Fortunately the most common default is for the ‘former message’ to be included after the ‘new message text’. In this situation we can try and eliminate most of this text automatically.  To do this we need to ensure that the outgoing message contains our ‘'special reply text string”. We can accomplish this by the use of a ‘hidden’ HTML tag at the start of the outgoing message as follows:

#-#- Text following text is ignored –#-#

 [The ‘#-#-‘ string (and its derived inverse) is a component setting and can be any desired unique string that is unlikely to be provided by the user in the usual course of business.]

We add this to the email template body and it has the advantage that it will not be displayed in the message due to the span style which hides the text. If the user replies and includes the received message contents in their reply it would be picked up by the message body checking code and can then be eliminated from the message stored in the component issue.

This will not of course eliminate all unnecessary text from the ‘email update message’ but it can go a long way towards reducing it. 

PHP Storm 9 available

We have been using the JetBrains PhpStorm application in our development for some years and knowing some of our readers also are interested we note that PhpStorm 9 is now finally released!

PhpStorm 9, the next big release of our IDE for PHP and web development, is already here. Make faster progress with PhpStorm 9’s refined editing and debugging experience, deeper code understanding, advanced productivity boosts, and complete support for remote development:

PHP Language & Editing Experience: Includes postfix code completion for PHP, partial PHP 7 support, advanced code understanding (Introduce parameter refactoring, advanced PHP type inference, support for persistent language injections, and new inspections and intentions), and other enhancements.

New Debugging Experience: Inline debugger for PHP and debugging workflow enhancements (php.ini can be opened from settings, and debugger config validation has been enhanced).

Remote Development: Remote edit, PHP Code Sniffer and Mess Detector via remote PHP interpreters.

Many improvements related to frameworks and tools.

A range of new and updated third-party plugins available in the PhpStorm plugin repository.

IP Mapping, clustering, refresh – performance impacts

ipmappingWe mentioned the other day about using HTML5 to determine a site visitors location and using it to be displayed upon a Google map. We are here looking at the various options that need to be considered when setting the parameters for the best ‘site impression’.  I have deliberately used the term ‘impression’ because most of the ‘work’ is actually performed by Javascript in the users browser. Each user will most likely be using a different machine, i.e. Desktop, laptop, tablet, smartphone etc., not to mention the different processors included in each type, and all these will different performance characteristics.  This each user’s impression of ‘how fast’ a site actually is, will be different, and this is without considering the impact of any network performance in a) obtaining the source data from the web site and b) the transfer of data to/from the Google mapping servers.

Continue reading

Experiences with HTML5 mapping

ipmappingWe have recently been updating our IP Mapping Joomla component to handle HTML5 geolocation detection and thought this may be of interest to others.

IP Mapping was originally designed with the aim of displaying IP addresses upon Google Maps and experience has shown that although it works well it is very reliant upon the accuracy of the data held by the various database and communication suppliers. The various supplied of the IP to location mapping vary considerably in the accuracy of the location information. We ourselves have been ‘located’ as being several hundred miles away from where we were physically located, depending upon which IP-location provider we were using and when we were determining the location. Whilst this may be adequate for some, for others it is a little bit hit and miss. I am thinking here of a ‘local’ village or town intending to serve the local neighbourhood, who desire to know how widespread their visitors are.

Continue reading

Benefits of a RAID configuration

We unfortunately suffered a hard disk failure over the weekend, which meant that a replacement was required.  We had been watching the ’Reallocated sector count’ rising on the specific disk for some time and it was in range of 2000 so were not totally surprised by the turn of events. One can run with a small number of reallocated sectors as the bad sectors will have been replaced with spares. Their Logical Block Addresses (LBA) having been reallocated to other physical sectors.  It is only when the number of spare physical sectors run out that one runs into real problems.   We also note that a couple of the other drives have a few reallocated sectors but these are in the low numbers and are not increasing so there is no need for concern at the moment.

Fortunately the disk was part of a RAID 6 configuration which meant that no data was lost and we could run in a degraded mode until such time as we could replace the disk.  We could even have survived a further failure, but fortunately that was not necessary.

The disk duly arrive and we plugged it in and restarted our QNAP system.  The drive was accepted and a RAID rebuilt immediately started.   It was as simple as that. No reconfiguration of need to do anything at all, just wait for the rebuilt to complete.  Simplicity itself.

This all helps justify our original decision, several years ago to purchase a QNAP system, and this is the first drive that we have had to replace, so it has paid for itself over the years.

In that time the drive technology has moved on, and our replacement drive, a Seagate Constellation ES,  whilst not being identical to the original, it being a later and ‘faster’ model, was a perfect match, and we note runs slightly cooler.

Tags:

Quicken UK Personal Finance Part 2

qw2k43In an earlier post on this subject, we earlier mentioned our experiences in starting to upgrade our UK version of Quicken. This continues with our discoveries with the US version.

Having managed to obtain a copy of the US 2015 version from one of our contacts in the United States, we tried installing it on a test machine.

The first problem was that it didn't like the fact that our test machine had the earlier UK version installed, so we uninstalled and installed the 2015 US version.  This went well but when we tried torun it it complained about our locale for NON-UNICODE programs and that it should be set to set to 'English (United States)'.  So we changed the locale and tried again.

This time it requires us to register the version.   Here comes the second problem in that one requires a US address to be able to register the program. OK we can do that, so using the address of our US contact we try and register.  Unfortunately we cannot get it to register. It also soon becomes apparent that because we are located in the UK, we cannot access the US Intuit site via a browser either.

Time for a review:   The classic way to get around the problem of access to a US site which is restricting itself to thier own country is to use a VPN (Virtual Private Network).  One such product is named 'Tunnelbear' which enables one to 'fool' the target web site to think that one is in the US (or UK).  This particular VPN has a 'free' option so we install this and manage to successfully register the product.

Now we try and read our 2004 UK Qdata file.  It immediately objects to the file and says that it cannot read non US files.

Searching the web we discover that this is also a 'well known' problem. It is suggested that an earlier version of the US version for 2004 is used to read the UK files.  We can obtain a version from the US Intuit web site and we install this. We then try to read our 'updated 2004 UK' QData file.  Again we get reported error saying it can not read non US files.  Perhaps we can use a QIF file, which is an export from our 2004 UK version.  This has mixed success.  First the accounts that get created by the Import are all set to use the US currency. We get around this by first just importing the accounts, then modifying them so that they have the UK currency.  Then we try importing the data again.  It states that it reads 15000 records but we can immediately see that the final account values are incorrect.  It is obvious that the import has not handled the transactions that are between the different accounts.  In addition the dates of the transactions it has imported are wrong in most cases.  A number (most?) seem to have dates with the year in the early part of the 20th centrury.  i.e 190x., and there appear to be a number of transactions missing.

Given the number of transactions we are dealing with it is going to be a long winded task to check, and change the incorrect transactions  and Import them it is back to the drawing board.

So time for another review.  We have our 2004 UK version working fine and it seems to handle all that we require, is it worth the pain in continuing.  The answer has to be a resound no, so for the time being we will stay as we are and perhaps, and I mean perhaps, we will look at it again later, but for now we will leave it as it is.

Go To Top

Joomla! Debug Console

Session

Profile Information

Memory Usage

Database Queries