Parsoid/Deployments/2022
Dec 13 - Dec 15: V0.17.0-a9 as part of 1.40.0-wmf.14
[edit]- Fix wikilink parsing issues on redlinks, T309024
- TOCData: Extract code from Parser.php::finalizeHeadings
- Protect against long match length in CHAR_REFS_REGEX, T322892
- Update wikimedia/idle-dom to 1.0.0
- html2wt debugging: Improve debug tracing to clarify stages
- Test and comments tweakings, T309024
- Improve test coverage for round trip testing.
- build: Updating qs to 6.11.0
- Fix red link handling with url that contains a fragment, T309024
- Improve error feedback in RoundTrip.js
Dec 6 - Dec 8: V0.17.0-a8 as part of 1.40.0-wmf.13
[edit]- Bump Parsoid HTML version to 2.7.0
- Add a new TOCData class to hold an array of SectionMetadata objects
- Add method to check if variant conversion is implemented for language, T318401
- Bump ecmaVersion to 2018
- build: Updating composer dependencies
- build: Updating mediawiki/mediawiki-phan-config to 0.12.0
- Make Transformhandler accept all Etags.
- Avoid self:: keyword as string in callables
- ests: Use more precise PHPUnit assertions
- Allow captions ending in px, T276675
- Remove references to 'mw-headline-anchor', T323773
Nov 29 - Dec 1: V0.17.0-a7 as part of 1.40.0-wmf.12
[edit]- Rename SectionMetaData -> SectionMetadata; other improvements
- Add a test for red links in templated references, T323479
- Remove redlink URL parameters, T323479
- Fixes data-id for "follow" refs
- tests: Replace assertEmpty with assertSame
- Don't require charset parameter in content-type matcher
- Fix bad assumption in AttributeExpander:splitTokens, T322557
- build: Migrate PHPUnit from 8.5 to 9.5, T320245
- Unbreak Parsoid CI: Fix lang variant test with changed expectation
- Utils::clone: Support limited DOMNode cloning during debugging, T323297
- Remove redlink normalization from tests, T309024
- Make tests compatible with redlinks, T309024
- Align redlink representation between Parsoid and the legacy parser, T309024
- Revert use of BCP 47 codes in Parsoid's Content-Language header, T323052
- Update e2e tests after Parsoid REST API changes in core, T322715
- Split methods to take/return BCP-47 language codes
- Add methods to convert to/from BCP-47 and MediaWiki-internal language codes
- Allow use of wikipeg 3.0.0 and PHP 8.1
- DataAccess: Fix string interpolation, T314096
- PageHandler: Fix string interpolation, T314096
- Document where Parsoid is expecting/exposing mediawiki-internal language codes, T320662
- Turn on phan warnings about the use of deprecated methods
- Split more language-related tests
- Remove Dodo from composer.json, T307618
- Document ordering info for the MigrateTempateMetas DOM pass.
- Followup on 9145f812: Add test case to prevent future regressions
Nov 10: V0.17.0-a6
[edit]In order to break a depdency cycle and fix CI, a branch was made from the v0.17.0-a5 release and a single patch added on top, which allows the use of the wikimedia/wikipeg 3.0.0 release (which in turn adds support for PHP 8.1). No other changes from v0.17.0-a5.
Nov 1 - Nov 3: V0.17.0-a5 for 1.40.0-wmf.8
[edit]- Revert 1e4bfa13: P-wrapping & MigrateTemplateMetas interact badly
- Followup to d934a6ee: Improve handling of mixed syntax quote tags
- ParserTests syncing: Update directions for integrated mode
- MigrateMeta Followups: Deal with foster box metas
- Clean up unreachable code in WTUtils.php
- Ensure DSR computation is accurate if an unclosed comment is present, T284693
- Move MigrateTemplateMeta pass higher up in the pipeline
- Remove comments about WTUtils::fosterCommentData
- Improve hoisting of template meta tags, T306003
Oct 25 - Oct 27: V0.17.0-a4 for 1.40.0-wmf.7
[edit]- ParserTests: Split off some more test files
- DOMNormalizer: Add missing diff-mark on merged node, T320319
- Remove "href missing in a" warning, T321281
- Sync parserTests with core
- More splitting of the parserTests file
- Enable wt2wt in integrated mode for core repo parser tests
- html2wt: Handle malformed data-mw part without crashing, T319079
- Add parentheses to clarify logical tests
- api-testing: e2e test for roundtrip-test in CI, T314244
- Sync parserTests with core
- Start splitting parserTests file into multiple files, T111604
- sync-parserTests.js: Support file renaming via 'oldPath' parameter
- Sync parserTests with core and all extensions
- Overhaul parser test file syncing
- Minor: Convert vars to consts in parserTest syncing scripts
- Sync *parserTests.text with core
- Enable integrated mode for media parser test files
- Followup to d790e6ff: Get rid of invalid check
- Drop php pre-7.4 compat code
- Fix PHP_VERSION typo
- Fix whitespace in Grammar.php
- Tokens: Rename dataAttribs to clearer dataParsoid name
- Update regression-testing.php to php 7.4
Oct 11 - Oct 13: V0.17.0-a3 for 1.40.0-wmf.5
[edit]- Do not generate annotation tokens on </> in templates, T319078
- Make extractPageBundle inner content to be arrays
- composer.json: Allow psr/container 2, T319055
Oct 4 - Oct 6: V0.17.0-a2 for 1.40.0-wmf.4
[edit]- Improvements to SectionMetaData class
- Revert "Disable the GC if running under PHP 7.2", T230861
- Bump minimum required PHP version to 7.4 in phan
- Bump minimum required PHP version to 7.4; minimum required MW to 1.40
- Mark extensions as core dom spec markup compatible
- Add 'outputHasCoreMwDomSpecMarkup' as option in extension registration, T309024
- Declare a first-class object for section metadata, T296025
- extension.json: Add license-name
- Fix review comments from previous patch, T186241
Sep 13 - Sep 15: V0.17.0-a1 for 1.40.0-wmf.1
[edit]Sep 12: V0.16.0-a21 for 1.39 RC
[edit]- Sync media parserTests with core
- Update extension API to let extensions add temp data on nodes
Aug 30 - Sept 1: V0.16.0-a20 as part of 1.39.0-wmf.27
[edit]- Restore cleanupFormattingTagFixup pass, T314059
- Improve handling of "empty" nodes
- TransformHandler: do not ignore ETags., T310710, T311819
- Add nofollow support to SiteConfig & disable test checks for nofollow, T186241
- Ensure that broken parser functions are still recognized as parser functions, T314524
- Switch uses of Assert:unreachable() to throwing an UnreachableException, T247868
Aug 16 - 18: V0.16.0-a19 as part of 1.39.0-wmf.25
[edit]- ParserTests/TestMode: move this class from core to Parsoid
- ParserTests/Test.php: don't warn if 'selser' undefined in test runner options
- Internationalization support for Parsoid, T309024
- Bump content version from 2.5.0 -> 2.6.0
- Add config support for nowiki stripping for ext args in #tag calls, T299103
- Update documentation for empty patches
- TestRunner: actually run html/parsoid+standalone tests
- rt-testing: Don't post original wikitext for non-selser html2wt
- Use class property to build trace prefixes in DOMRangeBuilder
- Don't verify template wrapping expectations in annotation wrapping, T307172
- Fix dumping feature: write to logger, not error_log
- Update CLI logging features to log to file
- Prevent traversing too far up the tree in computeListEltWidth, T313105
- Log when active formatting elements are reopened inside figures, T314059
- Remove unnecessary element assertions in ComputeDSR
- Add loop and muted attributes to media elements, T308230
- Update baseconfigs
Jul 26 - Jul 28: V0.16.0-a17 as part of 1.39.0-wmf.22
[edit]- Followup on d28f22a2: Additional fixes
- Additional normalization of HTML to improve diff classification
- parserTests.txt: Add test case for piped link using |, T290526
- e2e test: cover etag handling with structure test, T311819
- Allow multivalues in rel attributes, T186241
- Remove isExtendedAnnotationWrapperTag utility
- Generalize DOMTraversing with state
- Implement T2529 sol-nl hack in Parsoid's native template expansion path, T241150
Jul 19 - Jul 21: V0.16.0-a16 as part of 1.39.0-wmf.21
[edit]- Implement relative redirects in parsoid extension., T311867
- composer.json: Allow wikimedia/scoped-callback v4.0.0
- Add type hints to match built-in interface for php 8.1, T289879
- Fix bug in Parsoid's Poem implementation, T311953
- Omit DisplaySpace processing inside pre nodes, T305658
- Use core's TransformHandler and ParsoidHandler, T310467
Jul 5 - Jul 7: V0.16.0-a15 as part of 1.39.0-wmf.19
[edit]- Revert "Only disable nested template expansion when not native"
- ParserTests: Update <tag> hook implementation to match core
- Fix edge case bug in TableFixups exposed by integrated parsertest run
- ParserTests: Redlink missing templates. Don't output empty spans.
- Fix bug in expanding and encapsulating params
- Only disable nested template expansion when not native
- TemplateHandler: Document decoupled template processing model
- Minor: Fix comment by removing stray JS code line
Jun 28 - Jun 30: V0.16.0-a14 as part of 1.39.0-wmf.18
[edit]- Bump output content version to 2.5.0, T273505
- Account for format being in gallery media options, T214601
- Use caption as alt on imgs when not present and caption isn't visible, T297443, T162360, T63566
- Consolidate media types to mw:File, T273505
Jun 21 - Jun 23: V0.16.0-a13 as part of 1.39.0-wmf.17
[edit]- Change DOMTraverser to stay within the subtree of the passed in node
- ParserTests: Make <indicator> mocks match output in integraged mode
- Avoid a call to get file info if there are no valid containers
- TestRunner.php: Update comments w/r/t interwiki setup
- TemplateHandler: Check explicitly for an empty string, T310667
- Stop recognizing noicon and noplayer media options, T134880
- Copy changes to core api tests to soften schema version check
- Make end-to-end API test more robust
Jun 14 - Jun 16: V0.16.0-a12 as part of 1.39.0-wmf.16
[edit]- Followup on e71d5197: Check for nulls before calling traverse
- Followup to 0042a88e: Fix broken usage of DOMTraverser, T310220
- Add forward compatibility to avoid serializing alt from caption, T297443, T162360, T63566
- Strip parser function prefix from content, T310025
- Get rid of nonsensical complexity in TemplateHandler, T303368
- Colon escape is already stripped in getWikiLinkTargetInfo
- Allow colon escape after interwiki prefix, T286425
- Assure we have a figure if $forceBlock is set when rendering media, T309901
Jun 7 - Jun 9: V0.16.0-a11 as part of 1.39.0-wmf.15
[edit]- WTS: Minor cleanup for code readability
- WTS: Remove a few unnecessary PORT-FIXME comments
- Only adjust for parser func safesubst in tpl target resolution, T309405
- Remove priority hints and element timing from test normalization, T308621, T308707
- Account for multiple active formatting elts reopened in media
May 31 - June 2: V0.16.0-a10 as part of 1.39.0-wmf.14
[edit]- StubMetadataCollector: fix "already set" test for MERGE_STRATEGY_WRITE_ONCE
- AutoLoader: use registerNamespaces instead of $psr4Namespaces
- Add forward compatibility to serialize mw:File, T273505
- Set title for tooltips if caption isn't visible, T297443, T108380
- SiteConfig: Clear extension cache on extension registration
May 24-26: V0.16.0-a9 as part of 1.39.0-wmf.13
[edit]- Improve robustness of tagging misnested link-in-link content, T307412
May 17-19: V0.16.0-a8 as part of 1.39.0-wmf.12
[edit]- Remove guard against missing figcaption
- Set a title attribute for files linking to pages
- Revert "Lint captions on inline media without an explicit alt options", T297443
- Hoisting references outside of links, T301293
- Resolve linter test port-fixme, T237463
- Use isset to eliminate warnings
- Cleanup variable naming and documentation for clarity
- Cleanup confusing halign setting
- Don't escape extension tags shadowing html5 tags if found in source, T300436
- Lint html stashed in data-mw of mw:Extension/references, T214994
- Only lint content defined by a specific ref, T301293, T214994
- Prioritize body->html over body->id when diffing refs
- Fix using sort order from data-mw when serializing template params, T304730
- Move escapeNowikiTags to WTSUtils
- Escape dom elements with extension tag names when serializing, T300436
April 26-28: V0.16.0-a7 as part of 1.39.0-wmf.9
[edit]- Use DOMRangeBuilder "top level range" algorithm for nested ranges, T306186
Not deployed separately: v0.16.0-a6
[edit]- Lint defined width for framed or manualthumb formats
- Add lint test for multiple media formats defined, T305641
- Drop sizing when serializing frame or manualthumb, T64938
- Manualthumb does not do automatic scaling, T305759, T64938
- Manualthumb is a format, like the others, T305641, T305759
- Stop using ParsoidServices
- Add check for bad files, T75581
- Apply manualthumb consistently across media types, T302114
- Don't rely on the link element to roundtrip the |page= option, T259931
April 12-14: V0.16.0-a5 as part of 1.39.0-wmf.7
[edit]- ParserTests: Don't normalize style attributes
- ParserTests: Bunch of minor tweaks / refactoring
- Test.php: Remove Env references + make a function public
- Run `npm audit fix`
- ParserTests: Add a normalizeWT helper to Test.php
- Followup to 11cde591: Update rt-testing node.js scripts as well
- Remove ServiceWiring and Config from Parsoid extension
- Don't strip mw:IndentPre meta tags till the very end, T305597
- Making Translate extended ranges uneditable - fixing bugs, T305066
- Revert "Adjust whiteline handling around table for extended annotation ranges"
- ParserTests: Extract HTML normalization to the Test.php class
- Lint invalid upright options, T216003
- Drop ip-utils 3.x compatibility, core is on 4.0.0 now
- Making Translate extended ranges uneditable, T305066
- Adjust whiteline handling around table for extended annotation ranges
April 9: v0.15.0-a27 on REL1_38 branch
[edit]Cherry-picked changes since v0.15.0-a26:
- Making Translate extended ranges uneditable - fixing bugs, T305066
- Making Translate extended ranges uneditable, T305066
- Followup to 29a92a69: Fix missing unwrap invocation in PWrap
- PWrap: Improve p-wrapping behavior to minimize DOM range expansion
- PWrap: Improve readability by renaming and documenting a helper
- Make the `null` value clearer in PWrap
- Revert "Add temporary ParsoidSiteConfigInit hook", T303029
April 5-7: V0.16.0-a4 as part of 1.39.0-wmf.6
[edit]- Add a class for unwrapping of optional nodes in dom pwrapper
- Additional simplification of PreHandler
- Fix off-by-1 DSR error by cleaning up and simplifying PreHandler, T277415
- Followup to 29a92a69: Fix missing unwrap invocation in PWrap
- PWrap: Improve p-wrapping behavior to minimize DOM range expansion
- Add base constructors for abstract Config types
- PWrap: Improve readability by renaming and documenting a helper
- Make the `null` value clearer in PWrap
- Log extra info if we're unable to get link source when bailing, T302092
- Use in_array instead of array_search, where possible
- Followup of bb14815b: Refine TemplateHandler::convertToString
March 30: V0.15.0-a26 on REL1_38 branch
[edit]Cherry-picked changes since v0.15.0-a25:
- Add a class on the span representing the broken media element, T304010, T270150
- Fix complaint that test did not perform any assertions
- Document known failures testing functionality
- Drawings are derived and therefore always need the thumbnail treatment
- Add html/parsoid sections for svg rendering by page language tests, T208620
- Add baseconfig for ruwiki and refresh existing baseconfigs
- Support rendering SVG based on page language, T208620
- Use the check from RevisionRecord::getContent, T273111
Mar 29-31: V0.16.0-a3 as part of 1.39.0-wmf.5
[edit]- Improve performance of getAutoUrlTerminatingChars
- Add badFile to standalone response, T75581
- Sync mediaParserTests with core
- Document timing metrics we collect
Mar 22-23: V0.16.0-a2 as part of 1.39.0-wmf.4
[edit]- Collect html2wt metrics on non-selser path as well
- Sync parser tests files with core, T295187
- Add a class on the span representing the broken media element, T304010, T270150
Mar 22: V0.16.0-a1 as part of 1.39.0-wmf.3
[edit]- Fix complaint that test did not perform any assertions
- Document known failures testing functionality
- Drawings are derived and therefore always need the thumbnail treatment
- Add html/parsoid sections for svg rendering by page language tests, T208620
- Add baseconfig for ruwiki and refresh existing baseconfigs
- Support rendering SVG based on page language, T208620
- Use the check from RevisionRecord::getContent, T273111
Also, the following patches, which were tagged in v0.15.0-a25 but not released:
- Create abstract marker interface for the PageConfigFactory type
- [take 2] Expose ParserOutput/ContentMetadataCollector via ParsoidExtensionAPI
Unreleased: v0.15.0-a25
[edit]This release was tagged in order to allow CI to test patches which moved portions of Parsoid to core.
- Create abstract marker interface for the PageConfigFactory type
- [take 2] Expose ParserOutput/ContentMetadataCollector via ParsoidExtensionAPI
Mar 15-17: V0.15.0-a24 as part of 1.38.0-wmf.26
[edit]- Allow wikimedia/object-factory ^4.0.0
- Env isn't needed to dump diff marks
- Fix dumping diff marks
- Remove seemingly unnecessary template target resolution
- Add a class for the return value of the template expand call
- Construct TemplateEncapsulator earlier and pass it around
- Split template expansion / preprocessing from encapsulation
Mar 8-10: V0.15.0-a23 as part of 1.38.0-wmf.25
[edit]- Rename frame method s/fetchArg/expandArg/
- PreHandler: s/MULTILINE_PRE/SOL_AFTER_PRE to better reflect state
- Run ParserBeforePreprocess hook in DataAccess::preprocessWikitext
- Fix Grammar to not assume that a Token is returned, T303057
- Do not break heading detection on annotations, T296412
- html2wt: Use info level for unactionable Cite logspam caused by CX usage
- DOMDiff: Don't attempt to mark DOM Fragments, T302885
- Remove obsolete comment from parserTests SiteConfig
- Make processExtensionModule consistent for ease of reading
- Fix unsetting native extension tags in parserTests
- AttributeExpander: tokensToString invalidates srcOffsets
- AttributeExpander: Update token attributes/tsr correctly after processing, T299374
- Add mw-file-description class on links to the file description page, T292657
- Followup to df3124f9: Fix bad refactoring
- PreHandler: s/popLastNL/pushLastNL
- PreHandler: Get rid of recursion in onAny handler
- Clean up TemplateHandler::resolveTemplateTarget
Mar 1-3: V0.15.0-a22 as part of 1.38.0-wmf.24
[edit]- Account for strtok boolean return type, T302093
- Change Config/* classes to be abstract classes instead of interfaces, T261161
- Move bulk of template arg expansion to the Frame class
- Add logging of nested annotation tag removals
- Add description links consistently
- Don't repeat request for width if file is missing
- Only define our services if they haven't already been moved to core, T302118
- Move sanitization to when we're actually setting an external link
Feb 22-24: V0.15.0-a21 as part of 1.38.0-wmf.23
[edit]- ContentMetadataCollector: Use PHP constants to store merge strategy strings
- Tweak signature of ContentMetadataCollector::addCategory()
- Revert part of 55da3090: Setting up template args exposes other bugs, T301948, T302007, T301986
- Benchmark read view content stripping, T272331
- Cleanup unnecessary data-parsoid="{}" in parser tests.
- Minor cleanup in TableFixups & WrapSectionsState for future DRYing
- Followup to 50461b53: Fixup code updating transclusion parts, T301524
- Tweak parameters for API requests to add more metadata
- Don't add synthetic magic words to the output
- Move ::appendToHead() into DOMUtils
- Move substTopLevelTemplates logic to Core/
Feb 15-17: V0.15.0-a20 as part of 1.38.0-wmf.22
[edit]- Minor improvements to the ContentMetadataCollector interface
- Move ContentModelHandler back to Core/
- Consolidate some files under src/Wikitext/
- Get rid of the ExtensionContentModelHandler
- Move ContentModelHandler stuff from Core/ to Config/
- SiteConfig handler caching: Use an instance variable vs a static
- Cache extension tag handlers
- Fix old wikitext escaping bug, T277592
- Add preprocessWikitext to ParsoidExtensionAPI
- Introduce Parsoid\Core\Wikitext to hold wikitext concepts / operations
- Don't add synthetic categories from template expansion either
- Ensure frame args are properly set up in the pipeline
- Get rid of unused noDataAccess() mode
- Reorganize code in TemplateHandler::onTemplate for clarity
- Don't add synthetic categories to output HTML
- Get rid of optionalNewlines production from the grammar added in 6f4cd078
- build: Updating dependencies
- Moving Translate annotation declaration to the Translate repository, T295170
Feb 8-10: V0.15.0-a19 as part of 1.38.0-wmf.21
[edit]- Update Parsoid's CoreMetadataCollector to reflect patches made to core
- If we aren't expanding templates don't bother with the template name
- Move convertToString outside of call to encapTokens
- Simplify TemplateHandler::convertAttribsToString
- Assert that we aren't expanding an already expanded token, T300393
- Minor tweak to function name in ParsoidExtensionAPI
- TemplateHandler: Simplify resolveTemplateTarget
- Augment the CompoundTemplateInfo to indicate if it came from a param
- Get rid of retry signal from TokenHandlerResult
- Don't retry AttributeExpansion on templated attribute tokens, T300393
- Minor tweaks to lookupArg
- Set pipeline opts for attribute expander used in TemplateHandler
- Improve output of regression-testing.php script
- Simplify bailTokens
Feb 1-3: V0.15.0-a18 as part of 1.38.0-wmf.20
[edit]- Add temporary ParsoidSiteConfigInit hook
- Allow wikimedia/ip-utils ^4.0.0
- Support lang= option for SVGs, T208620
- Better handling of templated annotation attributes, T299375
- Initialize 'inTemplate' pipeline option always and fix usages, T226437
- ParserPipelineFactory: Remove duplicate call to defaultOptions
Jan 25-27: V0.15.0-a17 as part of 1.38.0-wmf.19
[edit]- WikiLinkHandler: Reprocess incorrect tokens to the right stage, T299185, T299454, T299457
- Use MediaWiki's object factory when instantiating extensions
- Shuttle tokens to the end of the stage
Jan 20: V0.15.0-a16 to beta
[edit]Parsoid v0.15.0-16 was tagged in order to push out CI changes for phab:T240775/gerrit:753824 It should be replaced with a thoroughly tested release build before the next train deploy.
- Simplify DOM class aliases., T240775
Jan 18-20: V0.15.0-a15 as part of 1.38.0-wmf.18
[edit]- Clarify a comment around special case for {{!}}, T299311
- Revert "Remove special case for {{!}} magic word", T299311
- Stop emitting "inline-media-caption" lints, T297443, T299302
- Fix parsing of interlanguage links with spaces, T156308
- WikitextSerializer: Fix production crasher, T299196
- Don't log wt2html metrics for linter requests + other tweaks
- Merge the first two token transform stages
Jan 11-14: V0.15.0-a14 as part of 1.38.0-wmf.17
[edit]Big wt2html changes
- Detect auto-insertion and tag deletion in RemexPipeline, T221145
- Followup to 819630e5: Match handling of mixed-syntax tags
- Followup to 819630e5: Make endTSR behave like tsr on mw:EndTag metas
- Followup to 819630e5: Handle autoInsertedFlags on tokens
- Partially mimic impact of mw:TSRMarker metas on MigrateTrailingNLs
Other notable changes
- Remove special case for {{!}} magic word
- TSP: Process retokenized tokens through TTM stages 1 & 2, T297922
- SiteConfig: Address old FIXME related to computing function synonyms
- TableFixups: Minor perf tweaks and cleanup, T298665
- TableFixups: Log error on reparse failure, don't crash
- Logging: Add pages with wt2html timePerKB > 500 ms to slow-parsoid log
- Add metrics to track time spent linting
- Add strict_type declaration to a number of files missing them
- Resolve a few more stale PORT-FIXMEs + update method docs
- Followup to 78e60013: Eliminate spurious semantic diff reports
- Replace usages of deprecated wfWikiID(), T298059
- Use ParserOptions::newFromAnon instead of ParserOptions::newCanonical
- Remove scrubWikitext as an option, T297298
- Lint captions on inline media without an explicit alt options, T297443