FYI Technical Site News (for those interested)

[gerryvz]

Small update of the Apache software today, from version 2.4.39 to 2.4.41. Release notes for the new version are here, and include a few bug-fixes for security issues: http://www.apache.org/dist/httpd/CHANGES_2.4.41

Also, due to some changes and bug fixes in the Wiki software, I am creating a small php file that allows me to "dump" (refresh) the Zend Opcode cache. This cache is associated with the php language that the forum software uses, and what Opcache is, is the abilty to store pre-compiled bytecode in shared memory. This stored code prevents the loading and parsing of php scripts (basically, the Xenforo forum software) every time it is requested, and uses the stored code instead. This reduces the load on the server and dramatically speeds up the forum display speed and associated operations.

There are some php scripts associated with the Wiki software that need to be "dumped" and replaced with new, bug-fixed scripts to remedy a couple of errors that are in the code in the Opcache.

None of this will affect forum operation at all, and will be fully transparent to everyone. It's just an operational exercise to force the server to use new, bug-fixed code.

Cheers,
Gerry


In the attached image below, which is using the Linux "nano" text editor program in the green-screen control terminal, you can see the php code that I created (it's only three lines) that will flush the Opcode cache on demand. The filename is in the green bar at the bottom (flush_cache.php) and I am getting ready to save the file to the server.

 

[gerryvz]

I am hearing that XenForo version 2.2 is supposed to be coming out during the month of August. We'll see. I will evaluate its new feature set and determine (based on early reports) whether it's a good upgrade to do in the short term or longer term. Chances are that with a "dot" rev of the software, it will "break" some of the add-ons and the skins.

 

[gerryvz]

After some tweaking, I updated the VaultWIki software today from version 4.1.0 Beta 2 Build 2, to Beta 3 Build 1. The developer is on medical leave for the next month or so, but he did a lot of work before heading off-line to squash a lot of bugs. Everything appears to be functional after the upgrade.

Release notes are here:


New Features

• Tab-less Page Style [lite: no]
• Use Prefixes in Area Titles [lite: no]
• Change Auto-Link Prefix Per Wiki Page [lite: no]
• Preview Changes in Mass Management Tools [lite: no]

Behavior Changes

• Re-enabled WYSIWYG editing for section edits [lite: no]
• Use forum progress indicator for AJAX
• Installer uses local resources over CDN resources [lite: no]
• Content-list stat should prefer non-zero values

Bug Fixes

• Content-list stat popup not working [in: XF >= 2.0]
• Category live-results obscured by lower elements [in: XF >= 2.0] [lite: no]
• Double-click to mark read not working [in: vB]
• User find-feed-content links to feed index [in: XF >= 2.0]
• Per-user feed-index shows entries by all users
• No link to feed-list from index when all existing feeds have no entries
• H=1 from template renders as H=2 [in: XF] [lite: no]
• Auto-TOC not directly before auto-collapsed heading [lite: no]
• FOOTNOTE BB-Code not fully rendered on viewed templates [in: XF] [lite: no]
• TOC BB-Code not affecting viewed templates [in: XF] [lite: no]
• Empty block if only line breaks before first H=1 [lite: no]
• Empty block heading for positioned DIV w/ no specified title [in: XF >= 2.0] [lite: no]
• Blank TOC entries on templates w/ H in INCLUDEONLY [in: XF >= 2.0] [lite: no]
• Duplicate breadcrumb for wiki navigation root page [in: XF >= 2.0]
• Fatal error sending approval PM notifications [in: XF >= 2.0]
• Fatal error in some JSON responses [in: XF >= 2.0]
• Wrong section editor contents if first section is only whitespace [lite: no]
• VaultWiki 4 import: infinite loop importing more than 500 wiki reputation entries [lite: no]
• Invalid action clicking help popup close-button
• Fatal error opening help popup sub-links [in: vB]
• Phishing protection cuts some editor drop-downs [in: XF >= 2.0]
• VaultWiki 4 import: database error importing database-stored attachment files [lite: no]
• VaultWiki 4 import: fatal error importing integrations [lite: no]
• VaultWiki 4 import: database error importing corrupt areas [lite: no]
• Importer errors when source defines invalid character sets [lite: no]
• Wiki-related CSS not included via platform CSS [in: XF >= 2.0]
• Canonical redirect redirects to wiki index if friendly URLs off
• Import redirect to missing page displays w/o redirecting [lite: no]
• Import redirect redirecting to forum index [in: XF >= 2.0] [lite: no]
• Link to find contributions by "Array" [in: XF >= 2.0] [lite: no]
• VaultWiki 3 regression: create date never used for last-update column [since: 4.0.0 Alpha 1]
• Javascript error: vw_ContentList_Item not defined [in: XF <= 2.0; vB]
• Broken HTML output of some sub-area listings [in: XF >= 2.0] [lite: no]
• Invalid feed link in wiki page feed-list widget [in: XF >= 2.0] [since: 4.1.0 Beta 1]
• Database error saving custom rules for discussions in lite upgrades [since: 4.1.0 Beta 2] [lite: no]
• Empty area choices for search criteria

Style Tweaks

• Border radius above wiki-related dialog footer [in: XF >= 2.0] [lite: no]
• Wiki-related admin dialogs max-height inconsistent w/ forum admin dialogs [in: XF >= 2.0]
• Too much top-padding for H=1 headings [in: XF >= 2.0] [lite: no]
• Missing label for Caption in File Browser [in: XF >= 2.0] [lite: no]
• Content-list stat display on narrow width [in: XF >= 2.0]
• Wrong padding for page-nav after wiki subscriptions [in: XF >= 2.0]
• Checkbox misaligned for alert-only wiki subscriptions [in: XF >= 2.0]
• Feed entry icon clickable area too large [in: XF >= 2.0]
• Existing disallowed titles hard to read due to padding [in: XF >= 2.0] [lite: no]
• Only show wiki user stats if greater than 0 [in: XF >= 2.0]
• Extra padding around area listings [in: XF >= 2.0]
• Broken table for result list in mass-management tools [in: vB]

 

[gerryvz]

We had a little hiccup with the Enhanced Search capability, that started late yesterday. Interesting dissection and analysis as to what happened.

The ElasticSearch engine that powers the search function is set to cease operating when system disk space hits 95%. That point came yesterday, and caused the system to start throwing (as it should have by design) errors.

Deleting some files from the system got us down to 87%, which is a comfortable enough margin to buy us a few weeks. I have more files that I can delete from the system, which will buy us more time. I need to have the system, for certain static access to stored files, default out to the SECOND virtual drive that is attached to the system. I'm studying how to do that now, but haven't put much time into it.

More system optimization is also in order, such as rotating system logs, which are taking up large amounts of memory (current syslog is 3.9 GB). Every little bit of room freed up, helps !!

Anyway, I'll provide updates as I get the issue solved with the drive access.

None of this affects forum operations in the least -- it's just about optimizing and balancing our system resources.

Cheers,
Gerry

 

[gerryvz]

By the way, I updated the Ubuntu version from 18.04.2 to 18.04.3, as well as php from 7.3.8-2 to 7.3.9-1 over the weekend.

This was only the second significant (but incremental) Ubuntu system upgrade since the forum operating system was first installed in late February. We will likely have 3-4 more small Ubuntu 18.04 upgrades between now and next spring (April, 2020), when Ubuntu will release Ubuntu 20.04. This will be a MAJOR upgrade for Ubuntu, which they do only every two years (note that the releases are date-coded ... 18.04 = April, 2018; 20.04 = April, 2020).

What is significant about the 18.04 and 20.04 releases is that they are "LTS" releases, meaning that they have Long-Term Support of 5 years from the date of release. So, 18.04 will be fully supported until April, 2023. 20.04 will be supported until April, 2025.

Ubuntu also does "interim" releases, in between the LTS release dates. This means that there is actually already an 18.10 release, and a 19.04 release, and there will be a 19.10 release coming in the near future. These "interim" releases are only supported for a short time, generally around 10 months. Interim releases are generally used as "proving grounds" for new features and capabilities.

For the purposes of our forum, we will stick to the "LTS" upgrade schedule for Ubuntu, meaning we will only use LTS upgrades, and of course the 'dot' revisions that are done within an LTS release.

For more information on the Ubuntu release cadence and philosophy, check these links.

LTS - Ubuntu Wiki

Ubuntu release cycle | Ubuntu

Overview of the Ubuntu release cycle - maintenance, support and security coverage, lifetime, upgrade paths, kernel versions and the range of editions and images published by Canonical.

ubuntu.com

Cheers,
Gerry

 

[gerryvz]

We had a little hiccup with the Enhanced Search capability, that started late yesterday. Interesting dissection and analysis as to what happened.

The ElasticSearch engine that powers the search function is set to cease operating when system disk space hits 95%. That point came yesterday, and caused the system to start throwing (as it should have by design) errors.

Deleting some files from the system got us down to 87%, which is a comfortable enough margin to buy us a few weeks. I have more files that I can delete from the system, which will buy us more time. I need to have the system, for certain static access to stored files, default out to the SECOND virtual drive that is attached to the system. I'm studying how to do that now, but haven't put much time into it.

More system optimization is also in order, such as rotating system logs, which are taking up large amounts of memory (current syslog is 3.9 GB). Every little bit of room freed up, helps !!

Anyway, I'll provide updates as I get the issue solved with the drive access.

None of this affects forum operations in the least -- it's just about optimizing and balancing our system resources.

Cheers,
Gerry

Some good news to report on the technical front.

First, removal of some additional old system logs has resulted in an additional 4 GB of storage space being freed up, bringing us down from 95% disk full, to 82%. The 4GB system log file was for the apache web server, and went back to late February when I first installed the new Ubuntu system in preparation for the migration of the old forum over to this one !!

Secondly, I have figured out how to access the second "block storage" disk drive, so that the forum will know to access certain files on that system rather than on the main drive, which is limited to only 80 GB (and which is now 82% full!). This will be done via what is called an "alias directive," which is a feature of the Apache web server. An alias directive specifies a path to a specific directory outside of the main forum storage drive -- in our case to a the second "block storage" drive.

The benefit of the "block storage" drive is that it is almost infinitely expandable, and is exceedingly cheap to run -- I believe it costs $0.10 per gigabyte per month. Right now the size of the block-storage drive is 75 GB, but in the future I will likely expand it to around 100 GB, and then as needed to 150 GB, or 200GB, or 500GB, whatever is needed. The main storage drive (which is 80 GB) is fixed at that size, as per the boundaries of our server plan.

After I get the initial "alias directive" up and running (I've already laid the foundation in terms of having the initial data migrated to the "block storage" drive), over time I am going to expand this dramatically. What I mean by this, is that eventually I will migrate ALL of the forum's attachments (images, PDFs, etc. that are found in our posts) to this "block storage" drive, instead of on the 80 GB main system drive where they are now.

This will "free up" the main system drive to basically hold the operating system and forum-related software only. All of the images will be on the attached "block storage" drive. You won't see any differences to forum operation at all - everything will be seamless and there wouldn't be any "down-time" associated with the switch over.

Speaking more in lay-person's terms, the technical equivalent of what I'm doing is as follows:

1. Plugging in a big, massive, hard drive to the USB port of your desktop computer
2. Formatting the new hard drive so that it is ready to hold files
3. Copying all of your digital images and photographs over to that new hard drive
4. Telling your photo program to go only to the new hard drive to access all of your stored digital images
5. Telling your photo program that the next time you sync your phone/digital camera to the computer, to send the new images ONLY to the new hard drive and NOT store them on your computer's internal hard drive
6. Deleting all of the images from your desktop computer's internal hard drive, to free up space

I'll keep everyone apprised on how it goes.

Right now I'm doing a very limited experiment on one small portion of the forum's files, which are not used very much, but take up A LOT of space on the main 80GB storage drive. If this is successful, then in the coming weeks I'll move all of the forum's image files to the new drive and we'll see how things go.

Just so you know, the current size of the "attachments" folder is 46.1 GB (which is well over one-half of the 80 GB system drive). It will free up quite a bit of space to move that 46+ GB lump of files over to the "block storage" drive!!

Cheers,
Gerry

 

[gerryvz]

I upgraded the Elasticsearch engine today, from version 6.8.2 to 6.8.3. It's generally working pretty well. The latest "cutting edge" version is 7.3.1, released on August 22, but I have not yet made the jump to 7.x, and don't need to anytime in the near future. Release notes for 6.8.3 are located here: Elasticsearch version 6.8.3 | Elasticsearch Reference [6.8] | Elastic

 

[gerryvz]

New version of the Linux kernel was released, version 4.15.0.69. I updated that from version 4.15.0.32 and issued a full system reboot at 4:39 AM EDT on Tuesday.

 

[gerryvz]

Had a little bit of a scare this morning....

Last night I did an Ubuntu update, a minor and almost daily occurrence whereby indvidual systems in the Ubuntu operating system are updated. Often these are 2-3 files, and tend to be security updates or small bug fixes, rather than major new functionality.

The nightmare scenario happened whereby one of the system files that was downloaded was somehow corrupted (on the server, not in the download process or on OUR server) and started throwing errors. I didn't see it until this morning, when I logged into the system back-end to ensure that everything was OK. There were errors showing both on the Linux server, as well as the XenForo control panel. Ruh-Roh !!

There were no obvious errors on the forum that I saw, but I did come across an error screen when I was trying to click on something. Double Ruh-Roh!

The XenForo errors were having to do with the php extension called "cURL," which is a piece of software that allows administrators (and the Ubuntu system) to type an ordinary URL into the Ubuntu command line and it will go out to that web page. This allows the Ubuntu system to manipulate/copy/download and do all kinds of things with the web page. In effect cURL "web-ifies" the Linux command line so that you don't need a browser to manipulate and handle web pages. cURL is an ESSENTIAL piece of software for XenForo to operate correctly!!

I had to track down the issue, and as I mentioned it was a corrupted file. So I had to find a correct file to replace it, download it, install it, activate it, and then place a Linux "hold" on that specific file so that future system updates don't replace it (again) with the bad file, until the issue is corrected. I am sure that will be within a day or two it'll be fixed.

The bug was reported here: Bug #1843507 “apt-get update to latest libidn2.so.0 causes multi...” : Bugs : libidn2 package : Ubuntu

But in the meantime, as I was trying to fix the problem, I had to reboot the entire server two times, and I had to (separately) restart the Apache web server a couple of times. This was all around 7:30 AM EDT today. So a couple of you may have experienced a non-responsive server/web site for a minute or two until both of the reboots completed. Generally, an Apache restart is not noticeable to users because it takes 1-2 seconds at most to happen.

Anyway, looks like things are error-free again. Interesting that a simple corrupted file can cause havoc. 99.9% of you wouldn't have seen/experienced it, but behind the scenes it was causing some real havoc with the Ubuntu operating system and the forum software !!

Cheers,
Gerry


System errors galore !!


XenForo errors galore !!










All is now well with the world !!

 

[gerryvz]

The Ubuntu bug report now says that the issue has been remedied, and the file has been updated on the server. I re-downloaded it and all is correct and working, so the temporary workaround ended up only being a couple of hours. All is well!

I also updated the Wiki software from 4.1.0 Beta 3 Build 1, to Beta 3 Build 2, on Monday. This killed a bunch of pesky bugs in the software. The developer is now back (early) from his medical leave.

 

[gerryvz]

Thursday around 6:30 PM EDT, we migrated from XenForo 2.1.3 to 2.1.4, which is a modest "bug-fix" point upgrade. Also upgraded the XenForo Media Gallery software from 2.1.3 to 2.1.4.

Everything went seamlessly, the forum was not down during or after the upgrade, and it required about 10 minutes for this "slipstream" upgrade.

A list of remedied issues with this new revision is below:

Some of the changes in XF 2.1.4 include:

• Fix some slightly over-zealous image removal code in the editor if image uploads are not supported.
• Improve performance of embed metadata rebuild.
• Fix regression relating to markdown being parsed in built-in block BBCodes
• Implement the russian ruble symbol
• Detect UTF-8 Byte Order Marker to prevent incorrect Windows-1252 fallback
• Standardize on \w for BB code tag matching, rather than a mishmash of [a-z0-9] and [a-z0-9_]
• Add support for user_id token in notices
• Don't enforce minimum tag requirements when using the quick lock/stick moderator actions
• Update draft last_update field when draft is updated.
• Make show current activity privacy checkbox dependent on show online status checkbox when editing a user
• Prevent guests from entering user upgrade checkout flow
• Use isIgnored methods in widget classes where available
• Ensure deleteChildAction option of XF:TreeStructured behaviour is passed through to child entities
• Workaround for image dragging bug in Firefox.
• Prevent additional content reports being considered a state change from Open->Open
• Show the 'unignore' button for users that can no longer be ignored
• Ensure tags containing quotes are escaped properly in the token input.
• Increase the maximum query length for the auto-completer to 15 characters (from 10).
• Allow IP hostnames in email addresses
• Update currency symbols for KZT and UAH
• Don't treat 6to4 IPv6 addresses as local
• Change the news feed item thread link to link to the thread rather than the post.
• Adjust in-editor vertical alignment to be the same as it is in the published post.
• Apply missing maxEntity field for warning_message column in various entity classes
• Prevent line break being inserted between two tables when rendering BBCode in editor
• Prevent caching of inline thread edit overlay to ensure accurate state
• Increase thread quick reply message container selector specificity
• When editing a phrase, only display the "Master value" row if the phrase exists in the "Master language".
• When using the schema manager and changing an existing column to a non-int type, remove the unsigned property if set.
• When rendering push templates, attempt to maintain the receiving user's style (or the site default).
• Move restoration of original language and style after push template render to finally block
• Delete reply ban alerts sent to unbanned user rather than from banning moderator
• Attempt to detect and block batch install of encrypted archives
• Make global '$xf' variable available to push templates
• Add workaround to prevent error when searching content in the admin panel based on value of custom field with numeric ID
• Handle string IDs in the abstract field map class when updating field associations.
• Fix issue with being able to delete emojis in the editor by reverting back to a previous version of the editor.
• Ensure the unapproved counts are recalculated correctly and take content visibility into account.
• Update inline_mod_actions event listener description to suggest using the global app object rather than public.
• Skip custom field max length checks for field types which tend to have a fixed length.
• Prevent unfurl block overlapping aligned image
• Fix issue causing high-numbered pages on bookmarks list to appear empty
• When an email address is changed, move user out of email bounced state if email confirmation is not required
• Wrap markForumReadByUser query in a separate transaction and attempt retry on deadlock
• Ensure message_count field is updated in cached entities when posts inserted/deleted
• Only hide the editor button for smilies if there are no smilies AND no option to use emoji.
• Remove defunct currency 'Lithuanian litas'
• Make \XF\Repository\AbstractField display group agnostic
• Fix n+1 queries on moderator list
• Correctly handle deleting directories with symlinks
• Run add-on onActiveChange method in a separate CLI process after transaction committed
• Use the term "login" rather than "log in" on the admin login form.
• Twitch VOD URLs were only embeddable if they had a start time in the URL. Allow base Twitch VOD URLs to be embedded.
• Standardize metadata fetcher header keys to be lower case. This allows Twitter profile URLs to be unfurled.
• Add new alert action for watched forum content alerts
• When indexing profile post comments, reduce the number of queries.
• When setting http client defaults, do not overwrite existing headers that have been passed in.
• For the new threads widget, adjust what is queried for in expanded mode.
• Remove the forced index hint for latest threads widget and add a new (post) date limit option.
• For some node types, display them as table-cells which will ensure vertical alignment.

Some of the changes in XFMG 2.1.4 include:

• Pass in a default username for the author name if the username is empty.
• Update gallery embed data retriever to use the same Metadata fetcher system as XF unfurling.
• When the media attachment handler is getting the attachment's container, ensure it eagerly fetches permissions and other relevant data to avoid additional queries later.
• Allow users to manage their own media/albums using inline moderation anywhere.
• Reimplement the ability to edit a media title and change the embed URL.

 

[gerryvz]

Yesterday (Wednesday) I did a small update to the Linux kernel, from version 5.0.0.27 to 5.0.0.29. Very modest update. This was basically a "fix" for three security vulnerabilities.

 

[gerryvz]

Installed a small update to the Wiki software, from version 4.1.0 Beta 3 Build 2 to 4.1.0 Beta 3 Build 3.

I had reported to the developer, two different bugs in the previous Build 2 software that were fixed in this Build 3 version.

 

[gerryvz]

This morning I updated the phpMyAdmin software (which allows administration, modification and backup of the MySQL database that back-ends the site) from version 4.9.0.1 to 4.9.1. This was a modest bug-fix update, and was released on 21-Sept-2019.

Here are the release notes:

Welcome to phpMyAdmin 4.9.1, a bugfix release. This is a regularly-schedule bugfix release that also includes some security hardening measures. We wish to point out that this also includes a routine fix for an issue that has been reported as CVE-2019-12922. The fix for this has been in our release queue to be part of this release, however it is the opinion of the team that the reported attack vector did not justify a separate release. This release includes fixes for many bugs, including: -Editing columns with CURRENT_TIMESTAMP for MySQL versions 8.0.13 and newer -Compatibility issues with PHP 8 -Export of GIS visualization -Enhanced descriptions for several collation types -Creating a user with a single quote in the password string -Unexpected quotes during import and export on text fields -Improvements to adding new tables to Designer -Fix an issue where an authenticated user could trigger heavy traffic between the database server and web server -Fix a weakness where an attacker, under certain conditions, working at the same time as an administrator is using the setup script, could delete a server from the setup script There are many, many more bug fixes thanks to the efforts of our developers, Google Summer of Code applicants, and other contributors. The phpMyAdmin team


The next upgrade to phpMyAdmin will likely be in 2020, and I will upgrade to version 5.0 (which is currently in the "alpha" stage of development).

 

[gerryvz]

Small Ubuntu kernel update was issued, so I updated it and did a reboot of the system this morning to make it live. System downtime was around one minute.

Cheers from Cliff Island and Great Diamond Island, Casco Bay, Maine

 

[gerryvz]

This morning php was updated from version 7.3.9-1 to 7.3.10-1.

This is a minor security and bug-fix update, but I upgraded it, and restarted the Apache web server for it to take effect. There was no system downtime or unavailability due to this "slipstream" upgrade.

This update also updated the OpenSSL software (which provides encryption for the site) from version 1.1.1c to 1.1.1d. This is only the second time since the site came up in March, that the OpenSSL software has been updated.

Release notes for php 7.3.10 are as follows:

PHP: PHP 7 ChangeLog

PHP is a popular general-purpose scripting language that powers everything from your blog to the most popular websites in the world.

www.php.net

 

[gerryvz]

Tonight I did an upgrade of the Wiki software from 4.1.0 Beta 3 Build 3, to 4.1.0 Beta 4 Build 2. Generally, everything went smoothly. The forum was off-line (but still running with the "DO NOT wig out or go postal" statement) for around 25 minutes while I did the update. This was mainly for bug fixes.

Cheers from Grants Pass, Oregon (about 60 miles north of the California / PRC border),
Gerry

 

[gerryvz]

Today was one of those periodic days when there were some significant system updates, mainly in the form of a Linux system kernel update.

The forum was off-line for approximately 90 seconds at 1:24 PM Eastern Daylight Time this afternoon, for a system reboot for the new Linux system files.

Specifically, the new kernel updated from version 5.0.0-29 to 5.0.0-31. Other kernel files updated from 4.15.0-65 to 4.15.0-66

The system had been "up" and running continuously for the past 15 days prior to the reboot.

 

[gerryvz]

I did a quick update of the Elasticsearch software (which powers our site's search engine and associated capability) from 6.8.3 to 6.8.4. Had to do a quick system reboot, so the forum was down for about one minute.



This version of the Elasticsearch engine was released earlier today.

Release notes on the new version of Elasticsearch:

Elasticsearch version 6.8.4 | Elasticsearch Guide [6.8] | Elastic

www.elastic.co

In the future, likely next year, we will be upgrading to version 7.4 or 7.5 of Elasticsearch, which is the current version.

 

[gerryvz]

This morning was a quick/modest update to php, from version 7.3.10-1 to 7.3.11-1. It was released on October 24, and is primarily a security release, with a few bugs fixed as well.



There was no system downtime or service interruption entailed with the upgrade -- only a quick restart of Apache, using the following Ubuntu command: service apache2 restart to load and initiate the new code.

If folks are interested, the release notes for this new version are located here:

PHP: PHP 7 ChangeLog

PHP is a popular general-purpose scripting language that powers everything from your blog to the most popular websites in the world.

www.php.net

 

[gerryvz]

Some interesting, generic statistics on our members.

This data is from the past 24 hours:

Browsers used:
Chrome = 44.4%
Edge = 1.4%
Firefox = 14.6%
Internet Explorer = 1.4%
Safari = 37.5%

Operating Systems:
Android mobile = 9%
Android tablet = 1.4%
iPad = 3.5%
iPhone = 38.1%
Linux = 0%
macOS = 4.1%
Windows = 43.8%

Computer Type:
Desktop = 68.7%
Mobile = 26.4%
Tablet = 5.9%

 

[Jlaa]

Some interesting, generic statistics on our members.

This data is from the past 24 hours:

Browsers used:
Chrome = 44.4%
Edge = 1.4%
Firefox = 14.6%
Internet Explorer = 1.4%
Safari = 37.5%

Operating Systems:
Android mobile = 9%
Android tablet = 1.4%
iPad = 3.5%
iPhone = 38.1%
Linux = 0%
macOS = 4.1%
Windows = 43.8%

Computer Type:
Desktop = 68.7%
Mobile = 26.4%
Tablet = 5.9%

Thanks, but this must be fake news. I don’t see any of @gsxr’s OS/2 or CPM or DOS machines represented here, nor his Mosaic browsers....

 

[gsxr]

Thanks, but this must be fake news. I don’t see any of @gsxr’s OS/2 or CPM or DOS machines represented here, nor his Mosaic browsers....

He must be running Edge on Win 10 inside a DOS virtual environment.

 

[gerryvz]

OS/2? That’s GUI-based.

Unfortunately, the green-screen character-based environments that he uses don’t show up in the results.

Nor does his PalmOS-based HP tablet.

 

[gerryvz]

In recent days, we have made a few modest updates to the forum software, including last week an update to the Wiki software (from version 4.1.0 Beta 4, Build 4 to 4.1.0 Beta 4, Build 5). This was primarily a bug-fix update.

Today, we updated the MySQL database software fro 5.7.27 to 5.7.28. This is a new "general availability" release of MySQL that was released on November 14th. Release notes for this version are posted here: MySQL :: MySQL 5.7 Release Notes :: Changes in MySQL 5.7.28 (2019-10-14, General Availability)

 

[gerryvz]

I had a little snag (error) in the command-line terminal when updating the database software, but was able to continue the installation of the database software and it seemed to complete OK. A check of the version confirmed that it installed correctly. Which reminds me, I need to do a restart of the database and Apache for good measure.... sigh. The forum was "down" for about 60 seconds with a white screen until I got the database install completed.




DONE.




New version confirmed:

 

[gerryvz]

Quick update to the Elasticsearch engine, bringing it from version 6.8.4 to 6.8.5

Elasticsearch version 6.8.5 | Elasticsearch Guide [6.8] | Elastic

www.elastic.co

A quick restart was all that was needed ( service elasticsearch restart ), and no forum downtime or service interruption occurred.

 

[gerryvz]

Quick update of the php language a couple of minutes ago, from 7.3.11-1 to 7.3.12-1. Minor update and simple/quick restart of Apache to get things going. Zero forum downtime.

Old version:



New version:




Release notes, for those interested. Mainly minor bug fixes.

PHP: PHP 7 ChangeLog

PHP is a popular general-purpose scripting language that powers everything from your blog to the most popular websites in the world.

www.php.net

Next up is a migration to php 7.4, which just hit general availability ("GA") a few days ago. I expect we won't migrate until after the New Year, although the current version of XenForo is compatible with it.

 

[gerryvz]

A new release of the Linux kernel was recently made generally available, and I updated the system files to incorporate it.

I will be rebooting the server so that it takes advantage of the new updates a around 12:00 noon EST, today (Monday). I expect the system to be down and unavailable for around 60-90 seconds while the server is rebooted.

The server has been continually "up" and available for the past 18 days.

Cheers,
Gerry

 

[Seangbadee]

Thanks for all you do

 

[gerryvz]

Today's software update was a minor-to-medium-scale operation, with a number of "point" software upgrades:

1. XenForo forum software upgrade from version 2.1.5 to 2.1.6
2. XenForo Enhanced Search upgrade from version 2.1.2 to 2.1.3
3. XenForo Media Gallery upgrade from version 2.1.5 to 2.1.6
4. php upgrade from 7.3.12-1 to 7.3.13-1
5. Elasticsearch engine software upgrade from version 6.8.5 to 6.8.6

 

[Seangbadee]

Thanks for the information

 

[gerryvz]

Last night, XenForo issued an "emergency" patch to the new software, called version 2.1.6 Patch 1, which fixed a couple of nasty bugs that had made it through their previous QA process. We weren't affected by the bugs, but they were deemed important enough to issue an updated patch/version of the software.

I updated the forum with the latest update this morning. Total forum time off-line: 12 minutes.

Cheers,
Gerry

 

[gerryvz]

Ever since the forum was first started back in late 2009, I've done periodic backups of the forum's database. Usually I have tried to do this about once a week (I've been better about this over the past couple of years). In the forum's early years, my backups may have been once every month or two.

I resolved to do better with this new forum. Since we migrated to XenForo back in March, I've been very good about doing a weekly backup of the database (a manual process that I initiate), as well as a daily, automatic backup of all of the forum's files on the server. This second forum-files backup I don't have to worry about, simply because it's part of the contract I have with Linode, the hosting company.

Today, I took an important step forward in ensuring even better security for the forum and its data: I implemented a daily, automated backup of the forum's database using the Linux cron function. Cron is a Linux command that automates pretty much any system function that you can think of - you can specify it to happen from every minute to once a year -- and any frequency in between.

The tough thing to get correct with a cron job is the actual syntax of the command, as even one character out of place will screw it up and it won't run at all. After a lot of trial and error, this is the cron command that is now backing up the forum's database on a daily basis. It is set to happen promptly at 1:00 AM Eastern Time, every night (the username and password are XXXXXX'd out, for security):

mysqldump --opt --default-character-set=utf8mb4 -u'XXXXXXXX' -p'XXXXXXXX' xenforo > /var/www/html/500eboard.co/mysql_backups/xenforo-backup-$(date +\%F).sql

This command will store a "rolling" seven days' worth of backups on the server. I will download a copy every week to my local hard drive, to join the many dozens of previous backups that I have archived, dating back to 2009.

Separately, another issue I've been concerned about is the ability to back up and archive all image files and attachments associated with the forum. I've also been hard at work on this small problem, as well. Thanks to the Linux rsync (remote sync) command, I am able to now do an automated comparison and download of all image files and document attachments from the forum's servers, to an identical folder on my local hard drive. This provides an extra layer of security for all of the many many thousands of attachments we have.

The Linux command to do this is:

rsync -av --delete --rsh=ssh 'XXXXXXXXXX@500eboard.co:/var/www/html/500eboard.co/public_html/forums/internal_data/attachments' /Volumes/Moe/internal_data

The rsync command is VERY powerful, and it works in this instance by going through each of the attachment/document files, determining which ones have been newly uploaded since the last syncrhonization, and then downloading ONLY the new files over the Internet to my local hard drive. Very cool stuff.

I've been meaning to get these processes into place for a long time, and am very happy that we now have these extra layers of security to ensure that the forum will never lose data.

Cheers,
Gerry

 

[Russell R Coe]

Ever since the forum was first started back in late 2009, I've done periodic backups of the forum's database. Usually I have tried to do this about once a week (I've been better about this over the past couple of years). In the forum's early years, my backups may have been once every month or two.

I resolved to do better with this new forum. Since we migrated to XenForo back in March, I've been very good about doing a weekly backup of the database (a manual process that I initiate), as well as a daily, automatic backup of all of the forum's files on the server. This second forum-files backup I don't have to worry about, simply because it's part of the contract I have with Linode, the hosting company.

Today, I took an important step forward in ensuring even better security for the forum and its data: I implemented a daily, automated backup of the forum's database using the Linux cron function. Cron is a Linux command that automates pretty much any system function that you can think of - you can specify it to happen from every minute to once a year -- and any frequency in between.

The tough thing to get correct with a cron job is the actual syntax of the command, as even one character out of place will screw it up and it won't run at all. After a lot of trial and error, this is the cron command that is now backing up the forum's database on a daily basis. It is set to happen promptly at 1:00 AM Eastern Time, every night (the username and password are XXXXXX'd out, for security):

mysqldump --opt --default-character-set=utf8mb4 -u'XXXXXXXX' -p'XXXXXXXX' xenforo > /var/www/html/500eboard.co/mysql_backups/xenforo-backup-$(date +\%F).sql

This command will store a "rolling" seven days' worth of backups on the server. I will download a copy every week to my local hard drive, to join the many dozens of previous backups that I have archived, dating back to 2009.

Separately, another issue I've been concerned about is the ability to back up and archive all image files and attachments associated with the forum. I've also been hard at work on this small problem, as well. Thanks to the Linux rsync (remote sync) command, I am able to now do an automated comparison and download of all image files and document attachments from the forum's servers, to an identical folder on my local hard drive. This provides an extra layer of security for all of the many many thousands of attachments we have.

The Linux command to do this is:

rsync -av --delete --rsh=ssh 'XXXXXXXXXX@500eboard.co:/var/www/html/500eboard.co/public_html/forums/internal_data/attachments' /Volumes/Moe/internal_data

The rsync command is VERY powerful, and it works in this instance by going through each of the attachment/document files, determining which ones have been newly uploaded since the last syncrhonization, and then downloading ONLY the new files over the Internet to my local hard drive. Very cool stuff.

I've been meaning to get these processes into place for a long time, and am very happy that we now have these extra layers of security to ensure that the forum will never lose data.

Cheers,
Gerry

Thank you Gerry

 

[TerryA]

Gerry,

I don’t understand anything that you are doing to keep us updated and secure BUT SINCERELY APPRECIATE all of the private time and effort you put forth for all of us 500Eboard members.

 

[gerryvz]

You can see the listing of all four of the individual files in the "backups" directory of the server. The one with the "2019-12-27" in the file name, and the "1:00" time stamp next to it, shows that the backup executed successfully in under one minute.

The other three files are "manual" backups that I did in recent days. The red color of the file names shows that the files are compressed (hence the ".gz" extension on the file name, meaning "gzip" which is a compression scheme).

You can see from the first, uncompressed file in the listing (no ".gz" on the end of the file name) that the total size of the database is more than 425 MB in size. The compressed size of the other database backups is only 104-105 MB ... a bit less than 25% of the size of the uncompressed database. This compression saves a LOT of space.

 

[gerryvz]

You guys can now see a full week's worth of the automated backups of the forum's database, with the filenames in red.

As you can see from the time-stamp, each backup is occurring like clockwork at exactly 1:00 AM EST, every night.

The file with the "2019-12-29" date in the listing, is a separate, once-weekly automatic backup job that is done at 1:15 AM every Sunday morning. I then automatically download this Sunday backup file to my hard drive (as I have always done). Again, you can see the 1:15 AM time-stamp on the file's creation.

Not only are these database backup files automatically backed up at 4AM every day (the complete collection of them in this directory) by the hosting provider, but as I said I download the "weekly" backup to my hard drive once a week as a separate precaution so as not to lose any data.



The forum has been up and running now for more than 30 days without a re-boot, which I think is pretty close to a record. Of course, the next major Linux system update will (eventually) require a reboot, but no major system files have been released/updated for the past couple of weeks due to the Holidays.

Happy 2020 !!

Cheers,
Gerry

 

[Seangbadee]

Happy New Year

 

[gerryvz]

A medium-scale update was made available today (after several weeks with no system updates due to the Holidays) for Ubuntu, which I have downloaded. I will be issuing a server reboot today around 4:30-4:45 PM EST (1:30-1:45 PM PST) to implement the new code.

I expect the forum will be down for around 60-90 seconds while the reboot takes place and the various services start back up.

It's been close to 35 days since the last system reboot....

 

[gerryvz]

System updates have been applied....reboot done. All is good. Forum downtime was under two minutes.

 

[gerryvz]

As a note to folks interested, I am in the early stages of getting everything together to do an update to the Java software running on the server. This is what the Elastic Search engine runs upon. Java has had some licensing changes, coming from its owner (and coincidentally my employer) Oracle Corp. during 2019. While I can still use it, there is a non-Oracle version of Java available that I may move to instead.

The issue is that the site is using the Java 8 software, which is past its end of life, and will actually become unsupported later this year in 2020. Which means that it won't get any security updates or other functionality improvements.

I will probably move to Java version 11, which is much more update to date and current.

I think Java is the last piece of software that back-ends the site that I've not updated since the new XenForo site went live on March 10th, 2019. So, it's time to get this software updated.

We are also a bit behind on the ElasticSearch software, as well, but it is relatively recent and not really that out of date. I may update that sometime in the coming couple of weeks, as well.

For any and all of these changes and updates discussed in this post, I don't expect to have any forum downtime or impact to the site's availability. It will just be a "slipstream" software update that no one should notice (if done correctly!).

Cheers,
Gerry

 

[gerryvz]

I updated the Linux foundational system files with new patches this morning, and that required a system reboot. The forum was down for approximately 90 seconds while this was happening. All is working well and everything is up to date.

This was a modest kernel upgrade from version 5.0.0 to 5.3.0.

On February 6, there will be the next major "point" upgrade to Ubuntu, going from 18.04.3 to 18.04.4. It is likely I'll upgrade to this in mid-February.

In April, the next major ("LTS" -- Long-Term Support) version of Ubuntu will be released. This will be called 20.04.0. This is a significant release because the LTS versions of Ubuntu are only released every two years -- in even years. Hence the current version that the site is using, 18.04.3, that we are on, was released in 2018, in April, and is the THIRD (soon to be FOURTH next month) bug-fix release.

We will upgrade to 20.04 later in April.

The LTS version of Ubuntu are important because they receive support for a full five years from the date of release (meaning that version 16.04 is still being very actively supported and updated). Ubuntu also releases "interim" versions of the operating system every six months (the latest interim release is 19.10, from October 2019), but these are only supported for around six months and then no longer updated or supported.

As a rule of thumb, it's best and most prudent to stick with the LTS releases of the operating system, which is my philosophy in terms of making sure this site is rock-solid.

 

[gerryvz]

Hi Folks,

I had to issue a full system reboot a few minutes ago, on an emergency basis, due to some system errors that cropped up when I upgraded the ElasticSearch engine from 6.8 to 7.5. The forum was down for about 90 seconds while the reboot happened, but it did solve the system errors that the new search engine software install created.

I also had to "re-index" the elasticsearch engine as part of the upgrade process.

The good thing now is that the search engine (which had been somewhat out of date, is now up to the current version, which offers a lot of performance improvements.

Thanks for your patience on this, and apologies if you got a minute or so of downtime. I sort of was sitting around here on the couch on a Friday night, and got a wild hair to upgrade the search engine.

 

[gerryvz]

As I've recently mentioned, the next major version of the Ubuntu operating system (code-name "Focal Fossa") will be released toward the end of April, 2020. This is an every-two-years type of occurrence.

If you are interested to see what will be included in this LTS release of Ubuntu 20.04, here's a blog post that is informative:

Ubuntu 20.04 Release Date & Planned Features (Updated)

Ubuntu 20.04 LTS was released on April 23, 2020, succeeding Ubuntu 19.10 as the latest stable release of this hugely popular Linux-based operating system

www.omgubuntu.co.uk

 

[gerryvz]

Tonight I did an upgrade of the php language from version 7.3.14 to 7.4.2. This took about two minutes to complete, and didn't involve a system reboot or any major disruption to things. php 7.4.2 was released on January 23, 2020, and is a feature and significant performance upgrade to 7.3. I am not expecting any problems with compatibility with any of the forum's add-ons with php 7.4. The XenForo software itself has been thoroughly tested with 7.4 and works perfectly.

The move to 7.4.x puts the forum from a technical standpoint into an excellent position for at least the next 5-7 years, although php 8, which is a MAJOR upgrade, will surely be released in the next 1-2 years.

More information about php 7.4 is here:

What’s New in PHP 7.4 (Features, Deprecations, Speed)

PHP 7.4 is coming with new features, deprecations, and a boost in performance. Check this in-depth overview of what's new in PHP 7.4!

kinsta.com

PHP: PHP 7 ChangeLog

PHP is a popular general-purpose scripting language that powers everything from your blog to the most popular websites in the world.

www.php.net

 

[gerryvz]

After an hour or so I reverted back to php 7.3.14, because the system threw a couple of very minor server errors from the Wiki software and an add-on, both of which haven’t been tested yet with php 7.4.

I’ll keep the forum on 7.3 until I know that everything has been tested with 7.4. Since 7.4 was initially released in November (we are on 7.4.2 already) it has not yet been tested for compatibility by all developers.



The Wiki software developer tells me that there is a brand-new version that is compatible with php 7.4, and that I should download and install it ... but I haven't seen it yet on his site. So I'll ask him where he has stashed it ;-) I see that it was released very late last night, so I'll download and install it. Sigh.

 

[gerryvz]

I have been generating a few errors over the weekend with the new version of the Wiki software, which is supposed to be a final beta ("Release Candidate 001 Build 1") after many months of testing. Several errors surfaced over the weekend, which I reported to the developer, with a fair bit of frustration, as these errors all started cropping up with this latest version, after things were quite good in recent months !!

I also consulted with a SECOND developer whom I trust, to get his thoughts on the errors. He confirmed that these errors are pretty much "for sure" flaws with the newest version of the Wiki software.

I don't think the errors were too apparent to anyone using the site or accessing the Wiki, but if it is one thing I **HATE** it is server error showing up in the admin logs. So I "disabled" the Wiki last night about 5:30 PM ET, so it was not available for use.

The developer notified me that he made several code changes last night and this morning, and I implemented those changes to help him test things out. Hopefully the errors are fixed. In any case, I will be monitoring things carefully in the coming days.

Feel free to help me out and use the wiki and read a few documents. If any errors come up, I'll see them in the system logs and report them (again) to the developer. You can't break the system and as it is, any errors that come up are going to be minor in nature anyway (fortunately).

Just wanted to give folks a heads up, just in case they do see the Wiki come and go, and wonder why. Some people notice these things, but most don't

 

[gerryvz]

My apologies, friends, as I had to do an unexpected "system reboot" today, around 11:00 AM EST.

The reason was that a new version of the Elasticsearch engine came out (7.6.0), and the system downloaded it and wanted to install it (upgrade from the previous 7.5.2). However, starting with version 7.6.0, Elasticsearch only runs on Java version 11 or higher, and the site only had Java version 8 installed (from a year ago when we first brought the new site up).

So I had to install the newest version of Java, ensure that it was all running OK and complete, and then issue a system reboot to complete the Elasticsearch upgrade.

I am happy to say that all is well, and the system was only down for approximately one minute. It had been up for 14+ days previous to this reboot.

Here's the old version of Java, as shown by the system:



And the new/updated version of Java:



And the updated version of Elasticsearch running:



Release Notes:

Elasticsearch version 7.6.0 | Elasticsearch Guide [7.16] | Elastic

www.elastic.co

 

[gerryvz]

I had to issue a full system reboot today, to restart the system because of two reasons:

1. a modest Ubuntu (Linux) kernel update, which happens about once a month, which requires a reboot to implement;

and 2. the system access log file had grown to around 4 GB in size, necessitating that I compress it and back it up, remove it, and start a new log file.

The system access logs detail every access to the system, and it is a file that continually grows over time. Over the past year, I had only compressed and removed the system log file one other time. A 4 GB file is a MASSIVE file, although it compressed down to around 350 MB. I downloaded the compressed file and then started a new log file from "scratch" -- rebooting the system started the system writing into the new log file.

Just in the last few hours, it's already grown from 0 to almost 2.75 MB !!!

The system log file is very important, because it provides a roadmap and fingerprints to go back (if needed) to examine who accessed the system, when, from where, and what they did (either authorized or unauthorized access).

Here's what the system log file looks like: