User:Daveh/2024 Work

Done Tasks

• VisualEditor -- Installed but won't work with UESP/SF Wikis due to MetaTemplate/Parsoid issues. -- Daveh ^(talk) 15:07, 21 February 2024 (UTC)
• ElasticSearch -- Installed v7.10.2 on search1. -- Daveh ^(talk) 15:07, 21 February 2024 (UTC)
• PCRE -- Updated the PCRE libraries on content1/2/3 from 8.32 to 8.45. -- Daveh (talk) 11:34 am, Today (UTC−5)
• PageSpeed -- Added simple special page for viewing stats. -- Daveh ^(talk) 16:18, 23 February 2024 (UTC)
• PHP 8.1 -- Installed on content1 for testing. -- Daveh ^(talk) 17:10, 28 February 2024 (UTC)

UESP/SF Wiki Specific

• Upgrade MediaWiki.
• VisualEditor on UESP (versions go back to 1.22)? Will not work with MetaTemplate currently

General

• Install PHP 8.1 for MW 1.39+.

• Actually 7.4.3+ works for all current MW versions up to 1.42 (Wikipedia is running 7.4).

• Update PCRE to 8.38 or later (for Semantic Wiki and Scribunto).
• New versions for ElasticSearch:

• 1.29.x - 1.30.x require Elasticsearch 5.3.x - 5.4.x.
• 1.33.x - 1.38.x require Elasticsearch 6.5.x - 6.8.x
• 1.39+ require Elasticsearch 7.10.2 (6.8.23+ using a compatibility layer)
• Currently running 5.3.3 and 6.8.23 on search1.

• Better DOS/rate limiting protection.
• Better Image File Server?

• Wikipedia uses Swift
• See https://wikitech.wikimedia.org/wiki/Swift
• Probably better to keep a separate NFS share with 1 or more file servers for now.

Capacity

• Files1

• Current: 150-250 req/sec (essentially 0 load)
• Max: 4k+ req/sec (appears to be limited by the 1Gbps network)

• Squid1

• Cache Hit Race: 62%
• Current Average: 70 req/sec
• Max: x25 (limited by network bandwidth)

• Content1/2/3

• Current Average: 20 req/sec
• Max: Estimated at around x5 (optimistic)

• db1/db2

• Unknown exactly but likely x10 or more with existing servers.

• Search1

• Current Average: 20 req/sec
• Max: 2600 req/sec (bandwidth limited)

• Estimated Capacity

• Max: x7 / 500 req/sec (on squid1 to content1/2)
• Depends on distribution of traffic increase. If there's more anonymous loads the squid1 cache will take more traffic and max will be higher. If more non-anonymous the content servers will take more traffic and max will be lower.
• If content3 is used for wiki traffic max increases a bit to x10 / 700 req/sec.

Fallout Wiki

• Hosting

• Currently running MW 1.37.2
• Running LiteSpeed with PHP 7.4.33
• Pygments 2.10.0 (from SyntaxHighlight)
• Uses h3/h2 for serving most connections/resources
• Host hivelocity.net

• Images

• images.fallout.wiki
• Cloudflare cached (mag-age 600s)

• Performance

• Typical server-side parsing is 0.5 to 2.0 sec.
• No obvious caching for anonymous views (there's the LiteSpeedCache extension but it does not appear to be working).
• Some sort of DOS protection.

• Stats

• Articles: 224,000 (64,000 content)
• Files: 124,000
• Page Edits: 4.4 million
• Users: 1000
• Semantic Values: 620,000
• Semantic Entities: 240,000
• Page Views (2023): ~120 million (300k/day)

• Extensions (not exhaustive)

• Semantic MediaWiki 4.0.1
• CreateWiki
• VisualEditor
• CodeEditor
• CodeMirror
• LiteSpeedCache (doesn't appear to be working)
• Scribunto
• TemplateData
• Variables
• TimedMediaHandler
• MultimediaViewer
• TitleKey

• Search (doesn't seem to use ElasticSearch)?

Notes

• Updating PCRE Library on CentOS7

    wget https://zenlayer.dl.sourceforge.net/project/pcre/pcre/8.45/pcre-8.45.tar.gz
    yum install readline-devel bzip2-devel
    ./configure --prefix=/usr --docdir=/usr/share/doc/pcre-8.45 --enable-unicode-properties --enable-pcre16 --enable-pcre32 --enable-pcregrep-libz --enable-pcregrep-libbz2 --enable-pcretest-libreadline -disable-static 
    make
    make install
    pcre-config --version

References

• MediaWiki Version Compatibility
• CirrusSearch
• Rate Limiting Apache
• Fallout Wiki Version
• Wikimedia Infrastructure