My Notes for Installing MediaWiki 1.28.0 on Ubuntu Server 16.04 LTS

Post Reply
User avatar
LHammonds
Site Admin
Site Admin
Posts: 772
Joined: Fri Jul 31, 2009 6:27 pm
Are you a filthy spam bot?: No
Location: Behind You
Contact:

My Notes for Installing MediaWiki 1.28.0 on Ubuntu Server 16.04 LTS

Post: # 481Post LHammonds
Mon Feb 13, 2017 5:55 pm

ON HOLD
WORK-IN-PROGRESS --------- WORK-IN-PROGRESS

Greetings and salutations,

I hope this thread will be helpful to those who follow in my foot steps as well as getting any advice based on what I have done / documented.

This is a Work-In-Progress topic so I will be updating this thread as I learn more about Ubuntu and MediaWiki.

To discuss this thread, please participate here: >> CREATE FORUM LINK <<

High-level overview

This thread will cover installation of an Apache web service and MediaWiki web site on a dedicated Ubuntu server and will utilize a separate dedicated database server. The server will be installed inside a virtual machine vSphere running on ESXi servers. Although there are some VMware-specific steps, they are very few and the majority of this documentation will work for other VMs such as VirtualBox or even directly installed onto a physical machine. Please excuse any ignorance on my part and if you have any advice on doing things better, please let me know. I love feedback and learning better ways of doing things!

After Ubuntu is installed and configured, Apache web server will be installed and configured. Next will be the installation and configuration of MediaWiki which will utilize an existing remote database server (MariaDB/MySQL). Some extensions will then be installed and tested.

The last step will cover some custom scripts to help automate tasks such as backing up, automatically growing the file system when free space is low, etc.

Tools utilized in this process

Helpful links

The list below are sources of information that helped me configure this system as well as some places that might be helpful to me later on as this process continues.

Assumptions

This documentation will need to make use of some very-specific information that will most-likely be different for each person / location. And as such, I will note some of these in this section. They will be highlighted in red throughout the document as a reminder that you should plug-in your own value rather than actually using my "place-holder" value.

Under no circumstance should you use the actual values I list below. They are place-holders for the real thing. This is just a checklist template you need to have answered before you start the install process.

Wherever you see RED in this document, you need to substitute it for what your company uses. Use the list below as a template you need to have answered before you continue.


  • Ubuntu Server name: srv-wiki
  • Internet domain: mydomain.com
  • Ubuntu Server IP address: 192.168.107.30
  • Ubuntu Admin ID: administrator
  • Ubuntu Admin Password: myadminpass
  • Email Server (remote): 192.168.107.25
  • MySQL Server (remote): 192.168.107.20
  • MySQL root Password: mysqlrootpass
  • MySQL wiki user: mediawikiuser
  • MySQL wiki Password: mediawikiuserpass
  • Windows Share ID: mediawikishare
  • Windows Share Password: mywikisharepass
I also assume the reader knows how to use the VI editor. If not, you will need to beef up your skill set or use a different editor in place of it.

User avatar
LHammonds
Site Admin
Site Admin
Posts: 772
Joined: Fri Jul 31, 2009 6:27 pm
Are you a filthy spam bot?: No
Location: Behind You
Contact:

Re: My Notes for Installing MediaWiki on Ubuntu Server 16.04

Post: # 482Post LHammonds
Mon Feb 13, 2017 5:56 pm

Install Ubuntu Server

The Ubuntu Server Long-Term Support (LTS) is free but we have the option of buy support and that is the main reason this server was selected.

The steps for setting up the base server are covered in this article: How to install and configure Ubuntu Server

It is assumed that the server was configured according to that article with the exceptions that the assumptions in red (variables above) are used instead of the assumptions in that document since we are building a database server.

User avatar
LHammonds
Site Admin
Site Admin
Posts: 772
Joined: Fri Jul 31, 2009 6:27 pm
Are you a filthy spam bot?: No
Location: Behind You
Contact:

Re: My Notes for Installing MediaWiki on Ubuntu Server 16.04

Post: # 483Post LHammonds
Mon Feb 13, 2017 5:58 pm

Configure MariaDB / MySQL

In this scenario, a dedicated and general-purpose database server already exists and it will be used to hold the application's database.
  1. Connect to the MariaDB/MySQL server using PuTTY.
  2. At the login prompt, login with your administrator account (administrator / myadminpass) and then temporarily grant yourself super user privileges by typing sudo su
  3. Type the following commands:
    mysql -u root -p Enter password: rootpass
    CREATE DATABASE mediawiki CHARACTER SET utf8 COLLATE utf8_bin; CREATE USER 'mediawikiuser'@'%' IDENTIFIED BY 'mediawikiuserpass'; GRANT ALL ON mediawiki.* TO 'mediawikiuser'@'%'; FLUSH PRIVILEGES; exit
    The above commands will allow the database account to connect from any machine from anywhere in the world. This might be OK if your database is not accessible outsite your local network or if your machine name changes or you have multiple servers that connect to the same database that use the same ID. You can make this more secure by specifying your application server when granting access. Make sure the database server will recognize the server name (via hosts file or DNS) or just use the IP address:
    CREATE USER 'mediawikiuser'@'srv-mediawiki' IDENTIFIED BY 'mediawikiuserpass'; GRANT ALL ON mediawiki.* TO 'mediawikiuser'@'srv-mediawiki';
    or
    CREATE USER 'mediawikiuser'@'192.168.107.30' IDENTIFIED BY 'mediawikiuserpass'; GRANT ALL ON mediawiki.* TO 'mediawikiuser'@'192.168.107.30';
    This will prevent anyone knowing the credentials from logging into the database from any other remote machine not specified in the grant command.

    If your application is running on the database server (typical on a developer machine / non-production scenario), create the user like this:
    CREATE USER 'mediawikiuser'@'localhost' IDENTIFIED BY 'mediawikiuserpass'; GRANT ALL ON mediawiki.* TO 'mediawikiuser'@'localhost';
    This will prevent anyone knowing the credentials from logging into the database from any other remote machine.

    If you mess anything up, you can remove the database and user by issuing these commands:
    DROP USER mediawikiuser; FLUSH PRIVILEGES; DROP DATABASE mediawiki;

User avatar
LHammonds
Site Admin
Site Admin
Posts: 772
Joined: Fri Jul 31, 2009 6:27 pm
Are you a filthy spam bot?: No
Location: Behind You
Contact:

Re: My Notes for Installing MediaWiki on Ubuntu Server 16.04

Post: # 484Post LHammonds
Mon Feb 13, 2017 5:59 pm

Install Apache Web Server

  1. Connect to the SRV-Wiki server using PuTTY.
  2. At the login prompt, login with your administrator account (administrator / myadminpass) and then temporarily grant yourself super user privileges by typing sudo su
  3. Type the following commands:

    Code: Select all

    apt -y install apache2 php7.0 libapache2-mod-php7.0
  4. If you plan on utilizing email functions (and who doesn't!!!), type the following commands:

    Code: Select all

    apt -y install php-pear
    pear install mail
    pear install Net_SMTP
    
  5. Once that is done, open a web browser and go to http://192.168.107.23 and you should see a web page that says "It works!"
Add Web Site Icon

If you want a custom icon to show up to the left of URL in the address bar, you need to follow these steps.
  1. Create a 16x16 image that is 16 colors (4-bit) with dimensions of 16x16 pixels and save it as a .BMP file called favicon.bmp
  2. Use your favorite icon editor to convert the BMP image to an ICO file. (e.g. IrfanView)
  3. Copy favicon.ico to \\192.168.107.23\share
  4. Connect to the SRV-Wiki server using PuTTY.
  5. At the login prompt, login with your administrator account (administrator / myadminpass) and then temporarily grant yourself super user privileges by typing sudo su
  6. Type the following commands:

    Code: Select all

    mv /srv/samba/share/favicon.ico /var/www/html/.
    chown www-data:root /var/www/html/favicon.ico
    chmod 0755 /var/www/html/favicon.ico
    
  7. Edit /etc/apache2/sites-available/000-default.conf and add the following line:
    AddType image/x-icon .ico
    Example:

    Code: Select all

    <VirtualHost *:80>
            ServerAdmin webmaster@localhost
            DocumentRoot /var/www/html
            ErrorLog ${APACHE_LOG_DIR}/error.log
            CustomLog ${APACHE_LOG_DIR}/access.log combined
            AddType image/x-icon .ico
    </VirtualHost>
  8. Restart the Apache web server by typing service apache2 restart
  9. Once that is done, open a web browser and go to http://192.168.107.23 and refresh the page (might have to hit CTRL+F5 to reload cached files). You should see your icon to the left of the URL in the address bar or in the page tab in Chrome.
EDIT: An alternative solution to creating the icon file yourself is to just upload your image to http://realfavicongenerator.net/ and let it generate the various icon files for you...assuming you upload an image large enough for it to process (such as 260x260 or higher)

User avatar
LHammonds
Site Admin
Site Admin
Posts: 772
Joined: Fri Jul 31, 2009 6:27 pm
Are you a filthy spam bot?: No
Location: Behind You
Contact:

Re: My Notes for Installing MediaWiki on Ubuntu Server 16.04

Post: # 485Post LHammonds
Mon Feb 13, 2017 6:01 pm

Install MediaWiki Prerequisites
  1. At the server console, type the following to install the additional modules and restart the web service :

    Code: Select all

    apt -y install php7.0-mysql php7.0-gd php7.0-intl php7.0-curl php7.0-mbstring imagemagick
    service apache2 restart
  2. To make sure the modules are loaded, we will create a phpinfo file:

    Code: Select all

    touch /var/www/html/phpinfo.php
    chown www-data:www-data /var/www/html/phpinfo.php
    chmod 0644 /var/www/html/phpinfo.php
    vi /var/www/html/phpinfo.php
  3. Now add the following to the file, save and close:

    Code: Select all

    <?php
    phpinfo();
    ?>
  4. Now open a web browser and go to http://192.168.107.23/phpinfo.php and verify that all the MediaWiki required modules are present.
  5. When satisfied everything is good, delete the test files:

    Code: Select all

    rm /var/www/html/index.html
    rm /var/www/html/phpinfo.php
Install MediaWiki

At the time of this writing, MediaWiki's current version is 1.28.1 so we will be downloading and installing that archive.

  1. Connect to the MediaWiki server using PuTTY.
  2. At the login prompt, login with your administrator account (administrator / myadminpass) and then temporarily grant yourself super user privilages by typing sudo su
  3. Type the following:

    Code: Select all

    cd /tmp
    wget https://releases.wikimedia.org/mediawiki/1.28/mediawiki-1.28.0.tar.gz
    tar -xzvf /tmp/media*.gz -C /var/www/html/ --strip-components=1
    chown www-data:root --recursive /var/www/html/*
    find /var/www/html/. -type d -exec chmod 755 '{}' \+
    find /var/www/html/. -type f -exec chmod 644 '{}' \+
    rm /tmp/media*.gz
    
  4. Tighten security on the "images" upload folder. Type the following:
    vi /etc/apache2/sites-available/000-default.conf
    Add the following:

    Code: Select all

    <Directory "/var/www/html/images">
        # Ignore .htaccess files
        AllowOverride None
        # Serve HTML as plaintext, don't execute SHTML
        AddType text/plain .html .htm .shtml .php
        # Don't run arbitrary PHP code.
        php_admin_flag engine off
        # If you've other scripting languages, disable them too.
      </Directory>
    Example configuration:

    Code: Select all

    <VirtualHost *:80>
      ServerAdmin webmaster@localhost
      DocumentRoot /var/www/html
      ErrorLog ${APACHE_LOG_DIR}/error.log
      CustomLog ${APACHE_LOG_DIR}/access.log combined
      AddType image/x-icon .ico
      <Directory "/var/www/html/images">
        # Ignore .htaccess files
        AllowOverride None
        # Serve HTML as plaintext, don't execute SHTML
        AddType text/plain .html .htm .shtml .php
        # Don't run arbitrary PHP code.
        php_admin_flag engine off
        # If you've other scripting languages, disable them too.
      </Directory>
    </VirtualHost>
  5. Restart the web service:

    Code: Select all

    service apache2 restart
  6. Open a browser and go to http://192.168.107.23/index.php, click on the "set up the wiki" link when the page loads.
  7. Installation Language - Click Continue to accept en-English
  8. Environmental Checks - Click Continue
  9. Connect to database - Set the following and click Continue:
    Database type: MySQL
    Database host: 192.168.107.20
    Database name: mediawiki
    Database table prefix:
    Database username: mediawikiuser
    Database password: mediawikiuserpassword
  10. Database settings - Set the following and click Continue:
    Check - Use the same account as for installation
    Storage engine: InnoDB
    Database character set: Binary
  11. Name - Set the following and click Continue:
    Name of wiki: MyWiki
    Project namespace: Same as the wiki name
    Your Name: JohnDoe
    Password: Hobo123!
    Password again: Hobo123!
    E-mail address: JohnDoe@mydomain.com
    Uncheck: Subscribe to the release announcements mailing list
    Check: Ask me more questions
  12. Options - Set the following and click Continue:
    User rights profile: Account creation required
    Copyright and license: No license footer
    EMAIL SETTINGS
    Check: Enable outbound e-mail
    Return e-mail address: no-reply@mydomain.com
    Check: Enable user-to-user e-mail
    Check: Enable user talk page notification
    Check: Enable watchlist notification
    Check: Enable e-mail authentication
    SKINS
    Check: CologneBlue
    Check: Modern
    Check: MonoBook (Use this skin as default)
    Check: Vector
    EXTENSIONS
    Check: Cite
    Check: CiteThisPage
    Check: ConfirmEdit
    Check: Gadgets
    Check: ImageMap
    Check: InputBox
    Uncheck: Interwiki (* NOTE: This is not needed if you are only hosting one wiki *)
    Uncheck: LocalisationUpdate
    Uncheck: Nuke
    Check: ParserFunctions
    Check: PdfHandler
    Check: Poem
    Check: Renameuser (* NOTE: Do not enable if you plan to use external LDAP authentication *)
    Check: SpamBlacklist
    Check: SyntaxHighlight_GeSHi
    Check: TitleBlacklist
    Check: WikiEditor
    IMAGES AND FILE UPLOADS
    Check: Enable file uploads
    Directory for deleted files: /var/www/html/images/deleted
    Logo URL: $wgResourceBasePath/resources/assets/mylogo.png
    Uncheck: Enable Instant Commons
    ADVANCED CONFIGURATION
    Uncheck: No caching
  13. Install Confirmation - Click Continue
  14. Install Done - Click Continue
  15. Save the LocalSettings.php to your PC and then transfer to \\192.168.107.23\share
  16. On the server console, type the following:

    Code: Select all

    mv /srv/samba/share/LocalSettings.php /var/www/html/.
    chown www-data:root /var/www/html/LocalSettings.php
    chmod 0600 /var/www/html/LocalSettings.php
    
  17. Increase the php filesize limit of uploaded files by editing /etc/php/7.0/apache2/php.ini and find upload_max_filesize and change it from 2M to 50M, then find post_max_size and change it from 8M to 51M
  18. Increase the wiki filesize limit of uploaded files by editing /var/www/html/LocalSettings.php and add the following line: $wgMaxUploadSize = '52428800'; (Reference)
  19. Allow various file types to be uploaded and certain file types to be blocked by editing /var/www/html/LocalSettings.php and add the following lines:

    Code: Select all

    # Allow the files to be uploaded with the following extensions:
    $wgFileExtensions[] = 'bmp';
    $wgFileExtensions[] = 'doc';
    $wgFileExtensions[] = 'docx';
    $wgFileExtensions[] = 'gif';
    $wgFileExtensions[] = 'jpg';
    $wgFileExtensions[] = 'jpeg';
    $wgFileExtensions[] = 'mp3';
    $wgFileExtensions[] = 'mpp';
    $wgFileExtensions[] = 'odt';
    $wgFileExtensions[] = 'ods';
    $wgFileExtensions[] = 'odp';
    $wgFileExtensions[] = 'odg';
    $wgFileExtensions[] = 'pdf';
    $wgFileExtensions[] = 'ppt';
    $wgFileExtensions[] = 'pptx';
    $wgFileExtensions[] = 'ps';
    $wgFileExtensions[] = 'png';
    $wgFileExtensions[] = 'tiff';
    $wgFileExtensions[] = 'xls';
    $wgFileExtensions[] = 'xlsx';
     
    # Block files from being uploaded with the following extensions:
    $wgFileBlacklist[] = 'exe';
    $wgFileBlacklist[] = 'php';
    $wgFileBlacklist[] = 'sh';
    $wgFileBlacklist[] = 'com';
    $wgFileBlacklist[] = 'vbs';
    $wgFileBlacklist[] = 'bat';
    $wgFileBlacklist[] = 'htm';
    $wgFileBlacklist[] = 'html';
    $wgFileBlacklist[] = 'js';
    
  20. Edit /var/www/html/LocalSettings.php and add the following line:

    Code: Select all

    $wgFavicon = "$wgScriptPath/favicon.ico";
  21. Create a logo image in PNG format that has the dimensions of 135 x 133 pixels. Save it to \\192.168.107.23\share\mylogo.png
  22. On the server console, type the following:

    Code: Select all

    mv /srv/samba/share/mylogo.png /var/www/html/resources/assets/.
    chown www-data:root /var/www/html/resources/assets/mylogo.png
    chmod 0644 /var/www/html/resources/assets/mylogo.png
    
  23. Edit /var/www/html/LocalSettings.php and add the following lines (preferably after the existing email variables):
    $wgSMTP = array( 'host' => "mail.mydomain.com", //could also be an IP address 'IDHost' => "mydomain.com", 'port' => 25, 'auth' => true, 'username' => "my_smtp_username", 'password' => "my_smtp_password" );
  24. To enable the WikiEditor by default, edit /var/www/html/LocalSettings.php and add the following lines (preferably right after WikiEditor extension line):

    Code: Select all

    $wgDefaultUserOptions['usebetatoolbar'] = 1;
    $wgDefaultUserOptions['usebetatoolbar-cgd'] = 1;
    $wgDefaultUserOptions['wikieditor-preview'] = 1;
    $wgDefaultUserOptions['forceeditsummary'] = 1;
    
  25. NOTE: To see a list of other preferences you can set by default, look at this page or run the following command: php /var/www/maintenance/userOptions.php --list
  26. To set configure some settings for the ConfirmEdit extension, edit /var/www/LocalSettings.php and add the following lines (preferably after the ConfirmEdit extension line):

    Code: Select all

    $wgCaptchaTriggers['edit']          = true;
    $wgCaptchaTriggers['create']        = true;
    $wgCaptchaTriggers['addurl']        = true;
    $wgCaptchaTriggers['createaccount'] = true;
    $wgCaptchaTriggers['badlogin']      = true;
    $wgGroupPermissions['emailconfirmed']['skipcaptcha'] = true;
    $ceAllowConfirmedEmail = true;
    
  27. Restart the Apache web server by typing service apache2 restart
  28. Visit http://192.168.107.23
  29. You should see your wiki page with your custom logo. Test your login ID that you created earlier.

User avatar
LHammonds
Site Admin
Site Admin
Posts: 772
Joined: Fri Jul 31, 2009 6:27 pm
Are you a filthy spam bot?: No
Location: Behind You
Contact:

Re: My Notes for Installing MediaWiki on Ubuntu Server 16.04

Post: # 486Post LHammonds
Mon Feb 13, 2017 6:02 pm

Install More Extensions for MediaWiki

HIGHLY OUTDATED SECTION

Normally, you will go to the Extension Distributor page to select your desired extension. Then you will pick which version of MediaWiki you are running (1.19.x in this case) and then obtain the snapshot that works with the specific MediaWiki site you are running. Download and copy the archive to \\192.168.107.23\share

Anytime you create new files on your website, you need to make sure the files and folders have the correct ownership and permission settings.

Run these commands each time you need to reset ownership/permissions:

Code: Select all

chown www-data:root --recursive /var/www/*
find /var/www/. -type d -exec chmod 755 '{}' \+
find /var/www/. -type f -exec chmod 644 '{}' \+
chmod 0600 /var/www/LocalSettings.php
chmod 0755 /var/www/favicon.ico
FlashMP3

  1. Copy the php code on the web page to a file called FlashMP3.php and copy it to \\192.168.107.23\share
  2. Download the standalone player archive. Extract audio-player.js and player.swf and copy them to \\192.168.107.23\share
  3. At the server console, type the following commands:

    Code: Select all

    mkdir /var/www/extensions/FlashMP3
    mv /srv/samba/share/FlashMP3.php /var/www/extensions/FlashMP3/.
    mv /srv/samba/share/audio-player.js /var/www/extensions/FlashMP3/.
    mv /srv/samba/share/player.swf /var/www/extensions/FlashMP3/.
    
  4. Reset ownership/permissions (see code at beginning of this section).
  5. Edit /var/www/LocalSettings.php and add the following to the bottom:

    Code: Select all

    require_once( "$IP/extensions/FlashMP3/FlashMP3.php" );
  6. Visit the version page to verify that the extension was installed and loaded: http://192.168.107.23/index.php/Special:Version
VideoFlash

  1. Copy the php code on the web page to a file called VideoFlash.php and copy it to \\192.168.107.23\share
  2. At the server console, type the following commands:

    Code: Select all

    mkdir /var/www/extensions/VideoFlash
    mv /srv/samba/share/VideoFlash.php /var/www/extensions/VideoFlash/.
  3. Reset ownership/permissions (see code at beginning of this section).
  4. Edit /var/www/LocalSettings.php and add the following to the bottom:

    Code: Select all

    require_once( "$IP/extensions/VideoFlash/VideoFlash.php" );
  5. Visit the version page to verify that the extension was installed and loaded: http://192.168.107.23/index.php/Special:Version
NOTE: The authors of this particular plugin did not include a version number and thus, none is displayed on the version page. You can correct this by editing the php file.

Change this:

Code: Select all

       'url' => 'http://www.mediawiki.org/wiki/Extension:VideoFlash'
);
To this:

Code: Select all

       'url' => 'http://www.mediawiki.org/wiki/Extension:VideoFlash',
       'version' => '1.0'
);

ContactPage

  1. Download ConfirmPage from the Extension Distributor and copy to the Ubuntu share
  2. On the server console, type the following:

    Code: Select all

    mv /srv/samba/share/ContactPage* /tmp/.
    tar -xzf /tmp/ContactPage* -C /var/www/extensions
  3. Reset ownership/permissions.
  4. Edit /var/www/LocalSettings.php and add the following to the bottom:

    Code: Select all

    require_once( "$IP/extensions/ContactPage/ContactPage.php" );
    $wgContactUser = 'Admin';
    $wgContactSender = $wgPasswordSender;
    $wgContactSenderName = 'Contact Form on ' . $wgSitename;
    $wgCaptchaTriggers['contactpage'] = true;
  5. Visit the version page to verify that the extension was installed and loaded: http://192.168.107.23/index.php/Special:Version
NOTE #1: The wgContactUser must reference a valid email-enabled wiki user in order to display the form properly.

NOTE #2: The authors of this particular plugin did not include a version number and thus, none is displayed on the version page. You can correct this by editing the php file.

Change this:

Code: Select all

       'url' => 'http://www.mediawiki.org/wiki/Extension:ContactPage',
       'descriptionmsg' => 'contactpage-desc',
To this:

Code: Select all

       'url' => 'http://www.mediawiki.org/wiki/Extension:ContactPage',
       'version' => '1.0.76660',
       'descriptionmsg' => 'contactpage-desc',
);
SyntaxHighlight GeSHi

  1. Download SyntaxHighlight GeSHi from the Extension Distributor and copy to the Ubuntu share
  2. On the server console, type the following:

    Code: Select all

    mv /srv/samba/share/Syntax* /tmp/.
    tar -xzf /tmp/Syntax* -C /var/www/extensions
    mv /var/www/extensions/SyntaxHighlight_GeSHi-master-f7617c9 /var/www/extensions/SyntaxHighlight_GeSHi
    
  3. Reset ownership/permissions.
  4. Edit /var/www/LocalSettings.php and add the following to the bottom:

    Code: Select all

    require_once( "$IP/extensions/SyntaxHighlight_GeSHi/SyntaxHighlight_GeSHi.php" );
  5. Visit the version page to verify that the extension was installed and loaded: http://192.168.107.23/index.php/Special:Version
Contribution Scores

  1. Download Contribution Scores from the Extension Distributor and copy to the Ubuntu share
  2. On the Ubuntu console, type the following:

    Code: Select all

    mv /srv/samba/share/Contrib* /tmp/.
    tar -xzf /tmp/Contrib* -C /var/www/extensions
  3. Reset ownership/permissions.
  4. Edit /var/www/LocalSettings.php and add the following to the bottom:

    Code: Select all

    require_once( "$IP/extensions/ContributionScores/ContributionScores.php" );
    $wgContribScoreIgnoreBots = true;          // Exclude Bots from the reporting - Can be omitted.
    $wgContribScoreIgnoreBlockedUsers = true;  // Exclude Blocked Users from the reporting - Can be omitted.
    $wgContribScoresUseRealName = true;        // Use real user names when available - Can be omitted. Only for MediaWiki 1.19 and later.
    $wgContribScoreDisableCache = false;       // Set to true to disable cache for parser function and inclusion of table.
    //Each array defines a report - 7,50 is "past 7 days" and "LIMIT 50" - Can be omitted.
    $wgContribScoreReports = array(
       array(7,50),
       array(30,50),
       array(0,50));
    
  5. CSS classes are used but not defined (this is optional but recommended). Edit /var/www/skins/common/commonElements.css and add the following lines:

    Code: Select all

    .contributionscores-wrapper    {  }
    .contributionscores-title      { background-color: #aaaaaa; margin-bottom: 0px; padding-left: .4em; }
    .contributionscores .header    { background-color: #cccccc; border-bottom: 1px solid #999999; font-weight: bold; }
    .contributionscores .odd       { background-color: #eeeeee; }
    .contributionscores .header td { padding-left: .2em; padding-right: .2em; }
    .contributionscores .content   { padding-left: .2em; padding-right: .2em; }
    
  6. Visit the version page to verify that the extension was installed and loaded: http://192.168.107.23/index.php/Special:Version

LDAP Authentication

All of our users are in a Microsoft Active Directory 2003 domain. These steps will allow MediaWiki to authenticate against the AD and auto-create a wiki account in the process of logging in.

References:
- LDAP Authentication Requirements
- LDAP Authentication Options
- How to enable LDAP over SSL with a third-party certification authority

Configure MediaWiki:
  1. Download LDAP Authentication from the Extension Distributor and copy to the Ubuntu share. NOTE: The current version is 1.2e and designed for MediaWiki 1.18.x but works in 1.19.0
  2. On the Ubuntu console, type the following:

    Code: Select all

    mv /srv/samba/share/LdapAuth* /tmp/.
    tar -xzf /tmp/LdapAuth* -C /var/www/extensions
    
  3. Reset ownership/permissions.
  4. Edit /var/www/LocalSettings.php and add the following to the bottom:
    require_once( "$IP/extensions/LdapAuthentication/LdapAuthentication.php" ); $wgAuth = new LdapAuthenticationPlugin(); $wgLDAPDomainNames = array("mydomain.com"); $wgLDAPServerNames = array("mydomain.com"=>"work.mydomain.com"); $wgLDAPUseLocal = false; $wgLDAPEncryptionType = array("mydomain.com"=>"ssl","mydomain.com"=>"tls"); $wgLDAPSearchStrings = array("mydomain.com"=>"work\\USER-NAME"); $wgLDAPDisableAutoCreate = array("mydomain.com"=>false); #LDAPDebug will show debug info # 0 (default) = disabled (best for production systems) # 1 = Will display non-sensitive info # 2 = Will display possibly sensitive info # 3 = Will display sensitive system info $wgLDAPDebug = 0;
  5. Visit the version page to verify that the extension was installed and loaded: http://192.168.107.23/index.php/Special:Version

Configure Ubuntu:
  1. Connect to the SRV-Wiki server using PuTTY.
  2. At the login prompt, login with your administrator account (administrator / myadminpass) and then temporarily grant yourself super user privilages by typing sudo su
  3. Install the LDAP library for PHP by typing install php5-ldap
  4. If your Active Directory domain has a certificate, type the following:

    Code: Select all

    openssl s_client -showcerts -connect [color=red]work.mydomain.com[/color]:636 > /usr/local/share/ca-certificates/cert.crt
    quit
  5. If your Active Directory domain does not have a certificate, use your web server:
    openssl s_client -showcerts -connect www.mydomain.com:443 > /usr/local/share/ca-certificates/cert.crt quit
  6. If you do not have a web server that has an SSL certificate, try google.com:443 instead.
  7. Edit /usr/local/share/cs-certificates/cert.crt
  8. Find the 1st certificate and delete everything above -----BEGIN CERTIFICATE-----
  9. Find the bottom of the 1st certificate and delete everything after -----END CERTIFICATE-----
  10. You should have something that looks like this:

    Code: Select all

    -----BEGIN CERTIFICATE-----
    MIIF0DCCBTmgAwIBAgIKFUj0TwAAAABS8DANBgkqhkiG9w0BAQUFADBGMQswCQYD
    VQQGEwJVUzETMBEGA1UEChMKR29vZ2xlIEluYzEiMCAGA1UEAxMZR29vZ2xlIElu
    dGVybmV0IEF1dGhvcml0eTAeFw0xMjA1MDIxMTA2NTdaFw0xMzA1MDIxMTE2NTda
    MGYxCzAJBgNVBAYTAlVTMRMwEQYDVQQIEwpDYWxpZm9ybmlhMRYwFAYDVQQHEw1N
    b3VudGFpbiBWaWV3MRMwEQYDVQQKEwpHb29nbGUgSW5jMRUwEwYDVQQDFAwqLmdv
    b2dsZS5jb20wgZ8wDQYJKoZIhvcNAQEBBQADgY0AMIGJAoGBAMGHpxE/UIg9t1Zs
    RcdR3PJ72quccUCxHzBLRrkBjexBL89+L+m9P/XFDDrXozTxoZlga2Hy0qkIdUv7
    vcaE16rl4vHhkOnvPLG683LFFOdRwWMuZeGUMXsqB3/GK7o2Ay3bzCrXmabv952f
    CzDa/vRuB5zBxlAtQl7UPfJfgy6lAgMBAAGjggOjMIIDnzAdBgNVHSUEFjAUBggr
    BgEFBQcDAQYIKwYBBQUHAwIwHQYDVR0OBBYEFDYKKSKamB4E8BXEAscAmzoM/9cS
    MB8GA1UdIwQYMBaAFL/AMOv1QxE+Z7qekfv8atrjaxIkMFsGA1UdHwRUMFIwUKBO
    oEyGSmh0dHA6Ly93d3cuZ3N0YXRpYy5jb20vR29vZ2xlSW50ZXJuZXRBdXRob3Jp
    dHkvR29vZ2xlSW50ZXJuZXRBdXRob3JpdHkuY3JsMGYGCCsGAQUFBwEBBFowWDBW
    BggrBgEFBQcwAoZKaHR0cDovL3d3dy5nc3RhdGljLmNvbS9Hb29nbGVJbnRlcm5l
    dEF1dGhvcml0eS9Hb29nbGVJbnRlcm5ldEF1dGhvcml0eS5jcnQwggJ3BgNVHREE
    ggJuMIICaoIMKi5nb29nbGUuY29tggpnb29nbGUuY29tgg0qLnlvdXR1YmUuY29t
    ggt5b3V0dWJlLmNvbYIWKi55b3V0dWJlLW5vY29va2llLmNvbYIIeW91dHUuYmWC
    CyoueXRpbWcuY29tgg8qLmdvb2dsZS5jb20uYnKCDiouZ29vZ2xlLmNvLmluggsq
    Lmdvb2dsZS5lc4IOKi5nb29nbGUuY28udWuCCyouZ29vZ2xlLmNhggsqLmdvb2ds
    ZS5mcoILKi5nb29nbGUucHSCCyouZ29vZ2xlLml0ggsqLmdvb2dsZS5kZYILKi5n
    b29nbGUuY2yCCyouZ29vZ2xlLnBsggsqLmdvb2dsZS5ubIIPKi5nb29nbGUuY29t
    LmF1gg4qLmdvb2dsZS5jby5qcIILKi5nb29nbGUuaHWCDyouZ29vZ2xlLmNvbS5t
    eIIPKi5nb29nbGUuY29tLmFygg8qLmdvb2dsZS5jb20uY2+CDyouZ29vZ2xlLmNv
    bS52boIPKi5nb29nbGUuY29tLnRygg0qLmFuZHJvaWQuY29tggthbmRyb2lkLmNv
    bYIUKi5nb29nbGVjb21tZXJjZS5jb22CECoudXJsLmdvb2dsZS5jb22CFiouZ29v
    Z2xldGFnbWFuYWdlci5jb22CFGdvb2dsZXRhZ21hbmFnZXIuY29tggwqLnVyY2hp
    bi5jb22CCnVyY2hpbi5jb22CFiouZ29vZ2xlLWFuYWx5dGljcy5jb22CFGdvb2ds
    ZS1hbmFseXRpY3MuY29tghIqLmNsb3VkLmdvb2dsZS5jb22CBmdvby5nbIIEZy5j
    bzANBgkqhkiG9w0BAQUFAAOBgQCyBzOsXXz5FhOxVcLeplWWss6sgJCvmAlzvhY/
    5YpdFsdKhzEmh13aogii7FGG3i+LH0i7dSlWy+QZ8v23lDbE6SGySQN2fusSThC8
    Wz1i/6D/kpJw90tGInJL/yvW9z60Hw1TUG6/heyYBHe2JhHa6CgqTyowRoaii1rA
    iQBF8w==
    -----END CERTIFICATE-----
  11. Save and close the file
  12. Verify that the certificate is valid by typing:

    Code: Select all

    openssl x509 -noout -text -in /usr/local/share/cs-certificates/cert.crt
  13. Now update the certificates by typing:

    Code: Select all

    update-ca-certificates
  14. Edit /etc/ldap/ldap.conf and make sure it contains the following lines:

    Code: Select all

    TLS_CACERTDIR   /etc/ssl/certs
    TLS_CACERT      /etc/ssl/certs/ca-certificates.crt
    
  15. Visit the login page and verify that a domain user can login (that does not have a mediawiki account): http://192.168.107.23/index.php?title=Special:UserLogin

User avatar
LHammonds
Site Admin
Site Admin
Posts: 772
Joined: Fri Jul 31, 2009 6:27 pm
Are you a filthy spam bot?: No
Location: Behind You
Contact:

Re: My Notes for Installing MediaWiki on Ubuntu Server 16.04

Post: # 487Post LHammonds
Mon Feb 13, 2017 6:03 pm

Scripting


Much of the solutions beyond this point involve scripts (programming snippets / automated commands).

In particular, they are Bash Scripts. I chose this due to its popularity and the fact it comes with Ubuntu. I try to make use of what comes with the system without requiring additional software / services unless they really add to the bottom line such as decreasing the time it takes for a process to run or to conserve storage and bandwidth usage.


MySQL Backup

My database server exports all databases and archives them on a normal schedule so any database I add to it such as the mediawiki database will get included automatically.

However, I created another script designed to run every minute looking for key files. If a specific file shows up on the samba share, it will trigger an immediate backup of the specified database. This will be helpful when scheduling the backup of the MediaWiki web files. When the web backup script run, it will place the database-specific file on the remote database server's samba share which will trigger the database backup to run immediately. The web backup script can trigger the remote database backup anytime it runs no matter if it is schedule via crontab or manually run.

Here is the script that runs every minute on the database server:

/var/scripts/prod/mysql-db-backup.sh

Code: Select all

#!/bin/bash
#############################################
## Name          : mysql-db-backup.sh
## Version       : 1.0
## Date          : 2012-05-14
## Author        : LHammonds
## Purpose       : Backup of a single database
## Compatibility : Verified on:
##     Ubuntu Server 10.04.3 LTS - 16.04.2 LTS
##     MySQL 5.1.62 - MariaDB 10.0.21
## Requirements  : p7zip-full (if ARCHIVEMETHOD=tar.7z), sendemail
## Run Frequency : As needed
## Exit Codes    : (if multiple errors, value is the addition of codes)
##    0 = success
##    1 = 7zip not installed
##    2 = archive failure
##    4 = archive purge failure
##    8 = configuration error
##   16 = mount warning
################ CHANGE LOG #################
## DATE       WHO WHAT WAS CHANGED
## ---------- --- ----------------------------
## 2012-05-14 LTH Created script.
#############################################

## Import common variables and functions. ##
source /var/scripts/common/standard.conf

LOGFILE="${TEMPDIR}/mysql-db-backup.log"
LOCKFILE="${TEMPDIR}/mysql-db-backup.lock"
TARGETDIR="${BACKUPDIR}/mysql-db"
OFFSITEBACKDIR="${OFFSITEDIR}/mysql-db"
ERRORFLAG=0

#######################################
##            FUNCTIONS              ##
#######################################
function f_PurgeOldestArchive()
{
  ## Purpose: Delete the oldest archive on the remote site.
  ## Return values:
  ##    0 = Success
  ##    1 = Cannot delete file
  ##    9 = Configuration error, path empty

  ## Variable Error Check. *
  if [ ${OFFSITEBACKDIR} = "" ]; then
    ## Make darn sure the path is not empty since we do NOT
    ## want to start purging files from a random location.
    echo "`date +%Y-%m-%d_%H:%M:%S` --- Purge error: OFFSITEBACKDIR site variable is empty!" >> ${LOGFILE}
    return 9
  fi
  ## Get the name of the oldest file.
  OLDESTFILE=`ls -1t ${OFFSITEBACKDIR} | tail -1`
  if [ "${OLDESTFILE}" = "" ]; then
    ## Error. Filename variable empty.
    echo "`date +%Y-%m-%d_%H:%M:%S` --- Purge error: OLDESTFILE variable is empty." >> ${LOGFILE}
    return 9
  else   
    FILESIZE=`ls -lak "${OFFSITEBACKDIR}/${OLDESTFILE}" | awk '{ print $5 }' | sed -e :a -e 's/\(.*[0-9]\)\([0-9]\{3\}\)/\1,\2/;ta'`
    echo "`date +%Y-%m-%d_%H:%M:%S` --- Purging old file: ${OFFSITEBACKDIR}/${OLDESTFILE}, Size = ${FILESIZE} kb" >> ${LOGFILE}
    rm "${OFFSITEBACKDIR}/${OLDESTFILE}"
    if [ -f "${OFFSITEBACKDIR}/${OLDESTFILE}" ]; then
      ## File still exists.  Return error.
      return 1
    else
      return 0
    fi
  fi
}

function f_cleanup()
{
  if [ -f ${LOCKFILE} ];then
    ## Remove lock file so other rsync jobs can run.
    rm ${LOCKFILE} 1>/dev/null 2>&1
  fi
  if [[ "${TARGETDIR}" != "" && "{TARGETDIR}" != "/" ]]; then
    ## Remove local backup files.
    rm -rf ${TARGETDIR}/*
  fi
  ## Email the result to the administrator.
  if [ ${ERRORFLAG} -eq 0 ]; then
    f_sendmail "MySQL DB Backup Success" "MySQL backup completed with no errors."
  else
    f_sendmail "MySQL DB Backup ERROR" "MySQL backup failed.  ERRORFLAG = ${ERRORFLAG}"
  fi
}

function f_emergencyexit()
{
  ## Purpose: Exit script as cleanly as possible.
  ## Parameter #1 = Error Code
  f_cleanup
  echo "`date +%Y-%m-%d_%H:%M:%S` - MySQL backup exit code: ${ERRORFLAG}" >> ${LOGFILE}
  exit $1
}

#######################################
##           MAIN PROGRAM            ##
#######################################

## Binaries ##
TAR="$(which tar)"
MY7ZIP="$(which 7za)"
MYSQL="$(which mysql)"
MYSQLDUMP="$(which mysqldump)"

if [ -f ${LOCKFILE} ]; then
  ## Program lock file detected.  Abort script.
  f_sendmail "MySQL DB Backup Aborted - Lock File" "This script tried to run but detected the lock file: ${LOCKFILE}\n\nPlease check to make sure the file does not remain when this script is not actually running."
  exit 1
else
  ## Create the lock file to ensure only one script is running at a time.
  echo "`date +%Y-%m-%d_%H:%M:%S` ${SCRIPTNAME}" > ${LOCKFILE}
fi

## Figure out which database will be backed up. Only one per run.
if [ -f "${SHAREDIR}/mediawiki" ]; then
  DATABASE="mediawiki"
  rm "${SHAREDIR}/mediawiki"
elif [ -f "${SHAREDIR}/phpbb" ]; then
  DATABASE="phpbb"
  rm "${SHAREDIR}/phpbb"
elif [ -f "${SHAREDIR}/wordpress" ]; then
  DATABASE="wordpress"
  rm "${SHAREDIR}/wordpress"
fi
if [[ "${DATABASE}" = "" ]]; then
  f_cleanup 0
  exit 0
fi

ARCHIVEFILE="`date +%Y-%m-%d-%H-%M`_mysql-db-${DATABASE}.${ARCHIVEMETHOD}"

echo "`date +%Y-%m-%d_%H:%M:%S` - MySQL ${DATABASE} backup started." >> ${LOGFILE}

## If the 7-Zip archive method is specified, make sure the package is installed.
if [ "${ARCHIVEMETHOD}" = "tar.7z" ]; then
  if [ ! -f "/usr/bin/7za" ]; then
    ## Required package (7-Zip) not installed.
    echo "`date +%Y-%m-%d_%H:%M:%S` - CRITICAL ERROR: 7-Zip package not installed.  Please install by typing 'aptitude -y install p7zip-full'" >> ${LOGFILE}
    ERRORFLAG=1
    f_emergencyexit ${ERRORFLAG}
  fi
fi

StartTime="$(date +%s)"

## Backup individual database.
${MYSQLDUMP} ${DATABASE} > ${TARGETDIR}/${DATABASE}.sql
## Create database sub-folder.
mkdir -p ${TARGETDIR}/${DATABASE}
## Export each table in the database individually.
for TABLE in `echo "show tables" | $MYSQL ${DATABASE}|grep -v Tables_in_`;
do
  FILE=${TARGETDIR}/${DATABASE}/${TABLE}.sql
  case "${TABLE}" in
    general_log)
      ${MYSQLDUMP} ${DATABASE} ${TABLE} --skip-lock-tables > ${FILE}
      ;;
    slow_log)
      ${MYSQLDUMP} ${DATABASE} ${TABLE} --skip-lock-tables > ${FILE}
      ;;
    *)
      ${MYSQLDUMP} ${DATABASE} ${TABLE} > ${FILE}
      ;;
  esac
done

## Compress the backup into a single file based on archive method specified.
echo "`date +%Y-%m-%d_%H:%M:%S` --- Compressing archive: ${TEMPDIR}/${ARCHIVEFILE}" >> ${LOGFILE}
case "${ARCHIVEMETHOD}" in
tar.7z)
  ${TAR} -cpf - ${TARGETDIR} | ${MY7ZIP} a -si -mx=9 -w${TEMPDIR} ${TEMPDIR}/${ARCHIVEFILE} 1>/dev/null 2>&1
  RETURNVALUE=$?
  ## Restore using one of the following commands (do not uncomment, only for notation):
  ## 7za x -so -w/tmp ${TEMPDIR}/${ARCHIVEFILE} | tar -C / -xf -
  ## 7za x -so -w/tmp ${TEMPDIR}/${ARCHIVEFILE} | tar -C ${TEMPDIR}/restore --strip-components=1 -xf -
  ;;
tgz)
  ${TAR} -cpzf ${TEMPDIR}/${ARCHIVEFILE} ${TARGETDIR} 1>/dev/null 2>&1
  RETURNVALUE=$?
  ## Restore using one of the following commands (do not uncomment, only for notation):
  ## tar -C / -xzf ${TEMPDIR}/${ARCHIVEFILE}
  ## tar -C ${TEMPDIR}/restore --strip-components=1 -xzf ${TEMPDIR}/${ARCHIVEFILE}
  ;;
*)
  ${TAR} -cpzf ${TEMPDIR}/${ARCHIVEFILE} ${TARGETDIR} 1>/dev/null 2>&1
  RETURNVALUE=$?
  ;;
esac

if [ ${RETURNVALUE} -ne 0 ]; then
  ## tar command failed.  Send warning email.
  f_sendmail "MySQL Backup Failure - tar" "tar failed with return value of ${RETURNVALUE}"
  ERRORFLAG=$((${ERRORFLAG} + 2))
fi

## Mount the remote folder. ##
f_mount

if [ ! -f ${OFFSITETESTFILE} ]; then
  ## Could not find expected file on remote site.  Assuming failed mount.
  ERRORFLAG=$((${ERRORFLAG} + 16))
  echo "`date +%Y-%m-%d_%H:%M:%S` --- ERROR: Cannot detect remote location: ${OFFSITETESTFILE}" >> ${LOGFILE}
  f_emergencyexit ${ERRORFLAG}
fi

FREESPACE=`df -k ${OFFSITEDIR} | grep ${OFFSITEDIR} | awk '{ print $3 }'`
BACKUPSIZE=`ls -lak "${TEMPDIR}/${ARCHIVEFILE}" | awk '{ print $5 }'`

## Make sure space is available on the remote server to copy the file.
if [ ${FREESPACE} -lt ${BACKUPSIZE} ]; then
  ## Not enough free space available.  Purge existing backups until there is room.
  ENOUGHSPACE=0
  while [ ${ENOUGHSPACE} -eq 0 ]
  do
    f_PurgeOldestArchive
    RETURNVALUE=$?
    case ${RETURNVALUE} in
    1)
      ## Cannot purge archives to free up space.  End program gracefully.
      echo "`date +%Y-%m-%d_%H:%M:%S` - ERROR: Not enough free space on ${OFFSITEBACKDIR} and cannot purge old archives.  Script aborted." >> ${LOGFILE}
      ## Stop and exit the script with an error code.
      ERRORFLAG=$((${ERRORFLAG} + 4))
      f_emergencyexit ${ERRORFLAG}
      ;;
    9)
      ## Configuration error, end program gracefully.
      echo "`date +%Y-%m-%d_%H:%M:%S` - ERROR: Configuration problem. Script aborted." >> ${LOGFILE}
      ## Stop and exit the script with an error code.
      ERRORFLAG=$((${ERRORFLAG} + 8))
      f_emergencyexit ${ERRORFLAG}
      ;;
    esac
    FREESPACE=`df -k ${OFFSITEDIR} | grep ${OFFSITEDIR} | awk '{ print $3 }'`
    if [ ${FREESPACE} -gt ${BACKUPSIZE} ]; then
      ## Enough space is now available.
      ENOUGHSPACE=1
    else
      ## Not enough space is available yet.
      ENOUGHSPACE=0
    fi
  done
fi

## Copy the backup to an offsite storage location.
echo "`date +%Y-%m-%d_%H:%M:%S` --- Copying archive file to offsite location." >> ${LOGFILE}
cp ${TEMPDIR}/${ARCHIVEFILE} ${OFFSITEBACKDIR}/${ARCHIVEFILE} 1>/dev/null 2>&1
if [ ! -f ${OFFSITEBACKDIR}/${ARCHIVEFILE} ]; then
  ## NON-FATAL ERROR: Copy command did not work.  Send email notification.
  echo "`date +%Y-%m-%d_%H:%M:%S` --- WARNING: Remote copy failed. ${OFFSITEBACKDIR}/${ARCHIVEFILE} does not exist!" >> ${LOGFILE}
  f_sendmail "MySQL Backup Failure - Remote Copy" "Remote copy failed. ${OFFSITEBACKDIR}/${ARCHIVEFILE} does not exist\n\nBackup file still remains in this location: ${HOSTNAME}:${TEMPDIR}/${ARCHIVEFILE}"
else
  ## Remove local copy of the compressed backup file
  rm ${TEMPDIR}/${ARCHIVEFILE}
fi

## Unmount the Windows shared folder.
f_umount

## Calculate total time for backup.
FinishTime="$(date +%s)"
ElapsedTime="$(expr ${FinishTime} - ${StartTime})"
Hours=$((${ElapsedTime} / 3600))
ElapsedTime=$((${ElapsedTime} - ${Hours} * 3600))
Minutes=$((${ElapsedTime} / 60))
Seconds=$((${ElapsedTime} - ${Minutes} * 60))

echo "`date +%Y-%m-%d_%H:%M:%S` --- Total backup time: ${Hours} hour(s) ${Minutes} minute(s) ${Seconds} second(s)" >> ${LOGFILE}

echo "`date +%Y-%m-%d_%H:%M:%S` - MySQL ${DATABASE} backup completed." >> ${LOGFILE}

## Perform cleanup routine.
f_cleanup
## Exit with the combined return code value.
exit ${ERRORFLAG}
Here is a sample of the log file:

/temp/mysql-db-backup.log

Code: Select all

2012-05-14_13:10:13 - MySQL backup exit code: 0
2012-05-14_13:10:33 - MySQL mediawiki backup started.
2012-05-14_13:10:34 --- Compressing archive: /var/temp/2012-05-14-13-10_mysql-db-mediawiki.tar.7z
2012-05-14_13:10:35 --- Copying archive file to offsite location.
2012-05-14_13:10:35 --- WARNING: Remote copy failed. /mnt/backup/mysql-db/2012-05-14-13-10_mysql-db-mediawiki.tar.7z does not exist!
2012-05-14_13:10:35 --- Total backup time: 0 hour(s) 0 minute(s) 2 second(s)
2012-05-14_13:10:35 - MySQL mediawiki backup completed.
2012-05-14_13:10:35 - MySQL backup exit code: 0
2012-05-14_13:12:37 - MySQL backup exit code: 0
2012-05-14_13:14:50 - MySQL mediawiki backup started.
2012-05-14_13:14:51 --- Compressing archive: /var/temp/2012-05-14-13-14_mysql-db-mediawiki.tar.7z
2012-05-14_13:14:55 --- Copying archive file to offsite location.
2012-05-14_13:14:56 --- Total backup time: 0 hour(s) 0 minute(s) 6 second(s)
2012-05-14_13:14:56 - MySQL mediawiki backup completed.
2012-05-14_13:15:13 - MySQL backup exit code: 0
2012-05-14_13:16:38 - MySQL mediawiki backup started.
2012-05-14_13:16:39 --- Compressing archive: /var/temp/2012-05-14-13-16_mysql-db-mediawiki.tar.7z
2012-05-14_13:16:40 --- Copying archive file to offsite location.
2012-05-14_13:16:41 --- Total backup time: 0 hour(s) 0 minute(s) 3 second(s)
2012-05-14_13:16:41 - MySQL mediawiki backup completed.
2012-05-14_13:16:47 - MySQL phpbb backup started.
2012-05-14_13:16:49 --- Compressing archive: /var/temp/2012-05-14-13-16_mysql-db-phpbb.tar.7z
2012-05-14_13:16:55 --- Copying archive file to offsite location.
2012-05-14_13:16:56 --- Total backup time: 0 hour(s) 0 minute(s) 9 second(s)
2012-05-14_13:16:56 - MySQL phpbb backup completed.
2012-05-14_13:16:59 - MySQL wordpress backup started.
2012-05-14_13:16:59 --- Compressing archive: /var/temp/2012-05-14-13-16_mysql-db-wordpress.tar.7z
2012-05-14_13:17:05 --- Copying archive file to offsite location.
2012-05-14_13:17:06 --- Total backup time: 0 hour(s) 0 minute(s) 7 second(s)
2012-05-14_13:17:06 - MySQL wordpress backup completed.

User avatar
LHammonds
Site Admin
Site Admin
Posts: 772
Joined: Fri Jul 31, 2009 6:27 pm
Are you a filthy spam bot?: No
Location: Behind You
Contact:

Re: My Notes for Installing MediaWiki on Ubuntu Server 16.04

Post: # 488Post LHammonds
Mon Feb 13, 2017 6:03 pm

MediaWiki Backup

There are various methods of backup that can be utilized but for this server, I will simply use RSync and tar.

RSync will mirror the files in the production folder to a local backup folder. The archive will be created from this backup folder.

Here are the folders I consider important that will be included in the backup:

Data Folders:
SRV-Wiki: /bak (which contains a mirror of /var/www/html)

Config Folders and files:
SRV-Wiki: /var/log/apache2/
SRV-Wiki: /etc/apache2/
SRV-Wiki: /etc/php/
SRV-Wiki: /etc/network/interfaces
SRV-Wiki: /etc/hosts

Remote Data:
SRV-MYSQL: MediaWiki Database

The backup of the web site will be fairly simplistic. However, since there are multiple servers involved, the backup will need to be synchronized for the web files, uploads and the remote database.

A crontab schedule will be used to schedule the local backups and when the backup is initiated on the local server, a special file will be sent to the remote database server so it will trigger a backup on the database that corresponds to this server (mediawiki).

The bulk of the data will be in /var/www and rsync will be used to copy any files from that folder to the backup folder. The benefit and beauty of using rsync in this way is that it will only copy what has changed since the last backup/sync. That means it will run VERY quickly even if your www folder has gigs of data in it. The other miscellaneous folders will be archived directly from their source location which will not be much in terms of size.

/var/scripts/prod/mediawiki-backup.sh

Code: Select all

#!/bin/bash
#############################################
## Name          : mediawiki-backup.sh
## Version       : 1.2
## Date          : 2017-02-14
## Author        : LHammonds
## Compatibility : Ubuntu Server 12.04 LTS - 16.04.2 LTS
## Purpose       : Backup web server while online.
## Run Frequency : One or multiple times per day.
## Exit Codes    : (if multiple errors, value is the addition of codes)
##   0 = Success
##   1 = rsync failure
##   2 = Archive creation failure
##   4 = Remote copy failure
##   8 = Cannot copy to MySQL
################ CHANGE LOG #################
## DATE       WHO WHAT WAS CHANGED
## ---------- --- ----------------------------
## 2012-05-14 LTH Created script.
## 2015-03-19 LTH Changed quotes used on SOURCES variable.
## 2017-02-14 LTH Changed mount to remote copy for MySQL.
#############################################

## Import common variables and functions. ##
source /var/scripts/common/standard.conf

LOGFILE="${LOGDIR}/${COMPANY}-mediawiki-backup.log"
TARGET="${BACKUPDIR}/mediawiki"
LOCKFILE="${TEMPDIR}/mediawiki-backup.lock"
ARCHIVEFILE="`date +%Y-%m-%d-%H-%M`_mediawiki-backup.${ARCHIVEMETHOD}"
SOURCES="/bak/mediawiki/www/ /var/log/apache2/ /etc/apache2/ /etc/php/ /etc/network/interfaces /etc/hosts"
ERRORFLAG=0
RETURNVALUE=0

#######################################
##            FUNCTIONS              ##
#######################################
function f_PurgeOldestArchive()
{
  ## Purpose: Delete the oldest archive on the remote site.
  ## Return values:
  ##    0 = Success
  ##    1 = Cannot delete file
  ##    9 = Configuration error, path empty

  ## Variable Error Check. *
  if [ ${OFFSITEBACKDIR} = "" ]; then
    ## Make darn sure the path is not empty since we do NOT
    ## want to start purging files from a random location.
    echo "`date +%Y-%m-%d_%H:%M:%S` --- Purge error: OFFSITEBACKDIR site variable is empty!" >> ${LOGFILE}
    return 9
  fi
  ## Get the name of the oldest file.
  OLDESTFILE=`ls -1t ${OFFSITEBACKDIR}/${HOSTNAME} | tail -1`
  if [ "${OLDESTFILE}" = "" ]; then
    ## Error. Filename variable empty.
    echo "`date +%Y-%m-%d_%H:%M:%S` --- Purge error: OLDESTFILE variable is empty." >> ${LOGFILE}
    return 9
  else   
    FILESIZE=`ls -lak "${OFFSITEBACKDIR}/${HOSTNAME}/${OLDESTFILE}" | awk '{ print $5 }' | sed -e :a -e 's/\(.*[0-9]\)\([0-9]\{3\}\)/\1,\2/;ta'`
    echo "`date +%Y-%m-%d_%H:%M:%S` --- Purging old file: ${OFFSITEBACKDIR}/${HOSTNAME}/${OLDESTFILE}, Size = ${FILESIZE} kb" >> ${LOGFILE}
    rm "${OFFSITEBACKDIR}/${HOSTNAME}/${OLDESTFILE}"
    if [ -f "${OFFSITEBACKDIR}/${HOSTNAME}/${OLDESTFILE}" ]; then
      ## File still exists.  Return error.
      return 1
    else
      return 0
    fi
  fi
}

function f_cleanup()
{
  if [ -f ${LOCKFILE} ];then
    ## Remove lock file so other backup jobs can run.
    rm "${LOCKFILE}" 1>/dev/null 2>&1
  fi
  echo "`date +%Y-%m-%d_%H:%M:%S` - MediaWiki backup exit code: ${ERRORFLAG}" >> ${LOGFILE}
}

#######################################
##           MAIN PROGRAM            ##
#######################################

## Binaries ##
TAR="$(which tar)"
MY7ZIP="$(which 7za)"
RSYNC="$(which rsync)"

if [ -f ${LOCKFILE} ]; then
  ## Script lock file detected.  Abort script.
  f_sendmail "MediaWiki Backup Aborted - Lock File" "This script tried to run but detected the lock file: ${LOCKFILE}\n\nPlease check to make sure the file does not remain when backup is not actually running."
  exit 1
else
  echo "`date +%Y-%m-%d_%H:%M:%S` ${SCRIPTNAME}" > ${LOCKFILE}
fi

StartTime="$(date +%s)"
echo "`date +%Y-%m-%d_%H:%M:%S` - Backup started." >> ${LOGFILE}

## Connect to the MySQL server to kick-off a remote database backup.
## NOTE: SSH keys must have been shared for this to work.
echo "Time to backup MediaWiki!" > /tmp/mediawiki
rcp /tmp/mediawiki root@srv-mysql:/srv/samba/share/.
RETURNVALUE=$?
if [ ${RETURNVALUE} -ne 0 ]; then
  ## Remote copy command did not work.
  ERRORFLAG=${ERRORFLAG} + 8
fi

## Output the version information to a text file which will be included in the backup.
if [ -f "${APPDIR}/version-info.txt" ]; then
  rm "${APPDIR}/version-info.txt"
fi
lsb_release -cd >> ${APPDIR}/version-info.txt
apache2 -v >> ${APPDIR}/version-info.txt
php -i >> ${APPDIR}/version-info.txt

## Check destination folder.  Create folder structure if not present.
if [ ! -d "${TARGET}" ]; then
  mkdir -p ${TARGET}
fi

## Synchronize files to backup folder.
${RSYNC} -apogHK --delete --exclude=*.pid ${APPDIR} ${TARGET} 1>/dev/null 2>&1
RETURNVALUE=$?
if [ ${RETURNVALUE} -ne 0 ]; then
  ## ERROR: Send email notification.
  echo "`date +%Y-%m-%d_%H:%M:%S` --- ERROR: Backup failed. ${APPDIR} -> ${TARGET}" >> ${LOGFILE}
  f_sendmail "Backup Failure - rsync" "ERROR: Backup failed. ${APPDIR} -> ${TARGET}, RETURN VALUE = ${RETURNVALUE}"
  ERRORFLAG=${ERRORFLAG} + 1
fi

## Compress the backup into a single file based on archive method specified.
echo "`date +%Y-%m-%d_%H:%M:%S` --- Compressing archive: ${TEMPDIR}/${ARCHIVEFILE}" >> ${LOGFILE}
case "${ARCHIVEMETHOD}" in
tar.7z)
  ## NOTE: Compression changed from 9(ultra) to 7 since it was blowing out on 512 MB RAM
  ${TAR} -cpf - ${SOURCES} | ${MY7ZIP} a -si -mx=7 -w${TEMPDIR} ${TEMPDIR}/${ARCHIVEFILE} 1>/dev/null 2>&1
  RETURNVALUE=$?
  ## Restore using one of the following commands (do not uncomment, only for notation):
  ## 7za x -so -w/tmp ${TEMPDIR}/${ARCHIVEFILE} | tar -C / -xf -
  ## 7za x -so -w/tmp ${TEMPDIR}/${ARCHIVEFILE} | tar -C ${TEMPDIR}/restore --strip-components=1 -xf -
  ;;
tgz)
  ${TAR} -cpzf ${TEMPDIR}/${ARCHIVEFILE} ${SOURCES} 1>/dev/null 2>&1
  RETURNVALUE=$?
  ## Restore using one of the following commands (do not uncomment, only for notation):
  ## tar -C / -xzf ${TEMPDIR}/${ARCHIVEFILE}
  ## tar -C ${TEMPDIR}/restore --strip-components=1 -xzf ${TEMPDIR}/${ARCHIVEFILE}
  ;;
*)
  ${TAR} -cpzf ${TEMPDIR}/${ARCHIVEFILE} ${SOURCES} 1>/dev/null 2>&1
  RETURNVALUE=$?
  ;;
esac

if [ ${RETURNVALUE} -ne 0 ]; then
  ## tar command failed.  Send warning email.
  f_sendmail "MediaWiki Backup Failure - tar" "tar failed with return value of ${RETURNVALUE}"
  ERRORFLAG=$((${ERRORFLAG} + 2))
fi

## Mount the remote folder. ##
f_mount

if [ ! -f ${OFFSITETESTFILE} ]; then
  ## Could not find expected file on remote site.  Assuming failed mount.
  ERRORFLAG=$((${ERRORFLAG} + 16))
  echo "`date +%Y-%m-%d_%H:%M:%S` --- ERROR: Cannot detect remote location: ${OFFSITETESTFILE}" >> ${LOGFILE}
  f_emergencyexit ${ERRORFLAG}
fi

FREESPACE=`df -k ${OFFSITEDIR} | grep ${OFFSITEDIR} | awk '{ print $4 }'`
BACKUPSIZE=`ls -lak "${TEMPDIR}/${ARCHIVEFILE}" | awk '{ print $5 }'`

## Make sure space is available on the remote server to copy the file.
if [[ ${FREESPACE} -lt ${BACKUPSIZE} ]]; then
  ## Not enough free space available.  Purge existing backups until there is room.
  ENOUGHSPACE=0
  while [ ${ENOUGHSPACE} -eq 0 ]
  do
    f_PurgeOldestArchive
    RETURNVALUE=$?
    case ${RETURNVALUE} in
    1)
      ## Cannot purge archives to free up space.  End program gracefully.
      echo "`date +%Y-%m-%d_%H:%M:%S` - ERROR: Not enough free space on ${OFFSITEBACKDIR} and cannot purge old archives.  Script aborted." >> ${LOGFILE}
      ## Stop and exit the script with an error code.
      ERRORFLAG=$((${ERRORFLAG} + 4))
      f_emergencyexit ${ERRORFLAG}
      ;;
    9)
      ## Configuration error, end program gracefully.
      echo "`date +%Y-%m-%d_%H:%M:%S` - ERROR: Configuration problem. Script aborted." >> ${LOGFILE}
      ## Stop and exit the script with an error code.
      ERRORFLAG=$((${ERRORFLAG} + 8))
      f_emergencyexit ${ERRORFLAG}
      ;;
    esac
    FREESPACE=`df -k ${OFFSITEDIR} | grep ${OFFSITEDIR} | awk '{ print $3 }'`
    if [ ${FREESPACE} -gt ${BACKUPSIZE} ]; then
      ## Enough space is now available.
      ENOUGHSPACE=1
    else
      ## Not enough space is available yet.
      ENOUGHSPACE=0
    fi
  done
fi

## Copy the backup to an offsite storage location.
echo "`date +%Y-%m-%d_%H:%M:%S` --- Copying archive file to offsite location." >> ${LOGFILE}
cp ${TEMPDIR}/${ARCHIVEFILE} ${OFFSITEDIR}/${HOSTNAME}/${ARCHIVEFILE} 1>/dev/null 2>&1
if [ ! -f ${OFFSITEDIR}/${HOSTNAME}/${ARCHIVEFILE} ]; then
  ## NON-FATAL ERROR: Copy command did not work.  Send email notification.
  echo "`date +%Y-%m-%d_%H:%M:%S` --- WARNING: Remote copy failed. ${OFFSITEDIR}/${HOSTNAME}/${ARCHIVEFILE} does not exist!" >> ${LOGFILE}
  f_sendmail "MediaWiki Backup Failure - Remote Copy" "Remote copy failed. ${OFFSITEDIR}/${HOSTNAME}/${ARCHIVEFILE} does not exist\n\nBackup file still remains in this location: ${HOSTNAME}:${TEMPDIR}/${ARCHIVEFILE}"
else
  ## Remove local copy of the compressed backup file
  rm "${TEMPDIR}/${ARCHIVEFILE}"
fi

## Unmount the Windows shared folder.
f_umount

## Calculate total time for backup.
FinishTime="$(date +%s)"
ElapsedTime="$(expr ${FinishTime} - ${StartTime})"
Hours=$((${ElapsedTime} / 3600))
ElapsedTime=$((${ElapsedTime} - ${Hours} * 3600))
Minutes=$((${ElapsedTime} / 60))
Seconds=$((${ElapsedTime} - ${Minutes} * 60))

echo "`date +%Y-%m-%d_%H:%M:%S` --- Total backup time: ${Hours} hour(s) ${Minutes} minute(s) ${Seconds} second(s)" >> ${LOGFILE}

echo "`date +%Y-%m-%d_%H:%M:%S` - MediaWiki backup completed." >> ${LOGFILE}

## Perform cleanup routine.
f_cleanup
## Exit with the combined return code value.
exit ${ERRORFLAG}
Here is some sample output from the log file.

/temp/mediawiki-backup.log

Code: Select all

2012-05-15_10:17:57 - Backup started.
2012-05-15_10:17:58 --- Compressing archive: /temp/2012-05-15-10-17_mediawiki-backup.tar.7z
2012-05-15_10:18:49 --- Copying archive file to offsite location.
2012-05-15_10:18:49 --- Total backup time: 0 hour(s) 0 minute(s) 52 second(s)
2012-05-15_10:18:49 - MediaWiki backup completed.
2012-05-15_10:18:49 - MediaWiki backup exit code: 0
2012-05-15_10:23:16 - Backup started.
2012-05-15_10:23:16 --- Compressing archive: /temp/2012-05-15-10-23_mediawiki-backup.tar.7z
2012-05-15_10:24:05 --- Copying archive file to offsite location.
2012-05-15_10:24:26 --- Total backup time: 0 hour(s) 1 minute(s) 10 second(s)
2012-05-15_10:24:26 - MediaWiki backup completed.
2012-05-15_10:24:26 - MediaWiki backup exit code: 0
2012-05-15_10:38:41 - Backup started.
2012-05-15_10:38:41 --- Compressing archive: /temp/2012-05-15-10-38_mediawiki-backup.tar.7z
2012-05-15_10:39:31 --- Copying archive file to offsite location.
2012-05-15_10:39:57 --- Total backup time: 0 hour(s) 1 minute(s) 16 second(s)
2012-05-15_10:39:57 - MediaWiki backup completed.
2012-05-15_10:39:57 - MediaWiki backup exit code: 0
2012-05-15_10:44:37 - Backup started.
2012-05-15_10:44:39 --- Compressing archive: /temp/2012-05-15-10-44_mediawiki-backup.tar.7z
2012-05-15_10:45:28 --- Copying archive file to offsite location.
2012-05-15_10:45:59 --- Total backup time: 0 hour(s) 1 minute(s) 22 second(s)
2012-05-15_10:45:59 - MediaWiki backup completed.
2012-05-15_10:45:59 - MediaWiki backup exit code: 0
Here is the top part of the version info file included in each backup.

/var/www/version-info.txt

Code: Select all

Description:    Ubuntu 12.04 LTS
Codename:    precise
Server version: Apache/2.2.22 (Ubuntu)
Server built:   Feb 13 2012 01:51:50
phpinfo()
PHP Version => 5.3.10-1ubuntu3.1

System => Linux srv-wiki 3.2.0-24-generic #37-Ubuntu SMP Wed Apr 25 08:43:22 UTC 2012 x86_64
Build Date => May  4 2012 02:18:29

User avatar
LHammonds
Site Admin
Site Admin
Posts: 772
Joined: Fri Jul 31, 2009 6:27 pm
Are you a filthy spam bot?: No
Location: Behind You
Contact:

Re: My Notes for Installing MediaWiki on Ubuntu Server 16.04

Post: # 489Post LHammonds
Mon Feb 13, 2017 6:03 pm

Crontab

To schedule the file/DB backups, add this to your crontab schedule on the MediaWiki server:

Code: Select all

0 22 * * * /var/scripts/prod/mediawiki-backup.sh > /dev/null 2>&1

User avatar
LHammonds
Site Admin
Site Admin
Posts: 772
Joined: Fri Jul 31, 2009 6:27 pm
Are you a filthy spam bot?: No
Location: Behind You
Contact:

Re: My Notes for Installing MediaWiki on Ubuntu Server 16.04

Post: # 490Post LHammonds
Tue Feb 14, 2017 9:27 am

Upgrading MediWiki

Please refer to the MediaWiki Upgrade instructions as they are your primary source of information.

The documentation below is what I did to upgrade a MediaWiki 1.25.1 instance to 1.28.0
  1. Make sure you have a recent backup of the MediaWiki site and its database.
  2. Setup the new version in a different folder by typing the following:

    Code: Select all

    mkdir -p /var/www/newwiki
    cd /tmp
    wget https://releases.wikimedia.org/mediawiki/1.28/mediawiki-1.28.0.tar.gz
    tar -xzvf /tmp/media*.gz -C /var/www/newwiki/ --strip-components=1
    rm /tmp/media*.gz
    
  3. Copy over the important bits from the current production version:

    Code: Select all

    cp -p /var/www/html/LocalSettings.php /var/www/newwiki/.
    cp -R -p /var/www/html/images/* /var/www/newwiki/images/.
    cp -p /var/www/html/resources/assets/*logo* /var/www/newwiki/resources/assets/.
  4. Ensure all permissions and ownership are correctly set:

    Code: Select all

    chown www-data:root --recursive /var/www/newwiki/*
    find /var/www/newwiki/. -type d -exec chmod 755 '{}' \+
    find /var/www/newwiki/. -type f -exec chmod 644 '{}' \+
  5. Swap the folders so the old code goes away and the new code becomes visible:

    Code: Select all

    mv /var/www/html /var/www/oldwiki
    mv /var/www/newwiki /var/www/html
  6. Run the upgrade code:

    Code: Select all

    cd /var/www/html/maintenance
    php update.php

Post Reply