Perseverance wins the day!
We recently investigated a problem upon our web site that impacted our Forum, and the ability to submit new topics or edit 'old' topics.
The problem was complicated in that there were no error or warning messages to provide any indications of what may be the cause. The most obvious symptom was that all of the problems were related and concerned the inability to submit 'form' data. i.e. The login process submits the entered details to the login process. The creation or editing of a forum post involves submitting 'entered data' to the post process.
Our immediate thought was that we were seeing a jQuery conflict, since we have seen similar problems in the past, again with no indication of the cause. We then proceeded to look at all the components, modules and plugins that we use upon the site but despite intensive searching we couldn't find anything that might be the cause, and there was certainly only one version of jQuery being loaded. Investigation also revealed that only forms submitted by the forum software were impacted. Very strange and seemed to indicate the forum software itself was at fault, but nobody had reported such a problem and when we tried it on another web site it worked perfectly. We also tried installing an 'older' version of the forum software and this didn't resolve the problem either.
One complication was that we were also having problems with our backup software, and found that it was not possible to restore a backup to enable to perform some tests upon a copy of the live site. We tried the software upon an earlier version of the site and it was working perfectly. This just made the problem more difficult to understand since that particular site was more likely to have a problem since it was more complex.
Once we have a solution to enable us to restore a current backup working correctly we performed a restore. This was interesting since the 'restore site' worked perfectly and didn't exhibit the problems seen on the 'live site'. This meant that the problem had to be caused by either a) the operating software upon the two platforms, b) the version of the underlying database, or c) the version of the PHP software used by the site. Looking at the technical requirements of the software all were being met but there had to be something getting in the way.
Upon a hosted platform there is very little if anything one can do about the operating system, so we were left with the PHP and/or database versions. We looked at the PHP versions and tried a number of versions including 5.4.44, 5.5.28 and 5.6.12, without any change in the problem. Changing the database version is more complex since it required the creation of a new database and a restore. The database versions upon both system was virtually the same being MySQL upon the live site and MariaDb upon the test site, although they were the same versions.
We therefore took the decision to try a system restore upon the live site with very little hope of a solution. To our surprise a system restore corrected the problem, which leads one to suspect that there was some corruption in the file system somewhere, but we currently have no idea exactly where at the moment. The most likely is that one or more of the forum software files was corrupt.
This was a long and exhausting investigation covering many days and resulting in delay to other activities. Looking back if our backups had been working correctly then we might have got there sooner, which makes one ask just how regularly does one test the backups. Obviously not enough has to be the answer in our case, even though we do test them periodically. I suspect we are not alone in not testing the backups sufficiently often.