DawoumWiki

Parsoid

Original article: mw:Parsoid

Parsoid는 위키텍스트와 HTML 사이를 앞으로 및 뒤로 변환하는 것을 허용하는 라이브러리입니다. 원래 응용프로그램은 자바스크립트 (Node.js를 사용)로 작성되었고 2012년 12월에 위키미디어 클러스터에서 작동했었습니다. 2019년에, Parsoid가 PHP로 포팅되었고, PHP 버전이 2019년 12월에 위키미디어 클러스터의 JS 버전을 대체했습니다. Parsoid는 궁극적으로 미디어위키의 현재 네이티브 파서를 대체하는 것을 목표로 핵심 미디어위키에 통합될 것입니다.

Parsoid
A bidirectional runtime wikitext parser. Converts back and forth between wikitext and HTML/XML DOM with RDFa.
Group: mw:Content Transform Team
Start: 2011-10
Team members: Arlo Breault, C.Scott Ananian, Isabelle Hurbain-Palatin, Mateus Santos, Shannon Bailey, Yiannis Giannelos
Backlog: #parsoid
Lead: Subramanya Sastry
Management: Sérgio Lopes
Updates: mw:Parsoid/Releases
Progress: mw:Parsoid/Deployments
Artist's impression of the Parsoid HTML5 + RDFa wiki runtime

Parsoid (PHP 버전)는 2020년 9월에 출시된 미디어위키 1.35에 기본적으로 번들로 제공됩니다. 비-위키미디어 설치에 대해, Parsoid/JS는 2021년 9월에서 미디어위키 1.31 (LTS)의 수명이 종료 때까지 지원됩니다.

Technical details

Parsoid는 미디어위키의 위키텍스트 구문과 동등한 HTML/RDFa 문서 모델 사이를 앞으로 및 뒤로 번역할 수 있는 응용프로그램으로 자동화된 처리와 풍부한 편집을 위한 향상된 지원을 제공합니다.

그것은 2012년부터 위키미디어 재판에서 한 팀에 의해 계속 개발 중에 있습니다. 그것은 현재 시각 편집기 확장, 토론 확장, 내용 번역과 기타 응용 프로그램에서 광범위하게 사용됩니다.

Parsoid는 흠없는 앞으로-및-뒤로 변환을 제공하기 위해, 즉, 정보 손실을 방지하고 역시 "더러운 차이(dirty diffs)"를 방지하기 위해 의도됩니다.

위키미디어 위키에서, 여러 응용프로그램에 대해, Parsoid는 현재 Parsoid에 의해 번역된 HTML을 저장하는 RESTBase 뒤에 프록시됩니다. RESTBase는 결국 미디어위키와 더 긴밀하게 통합된 캐시로 대체될 것으로 예상됩니다.

전체 프로젝트의 자세한 내용에 대해, 2013년 3월로부터 이 블로그 게시물을 참조하십시오. 사용 중인 HTML 모델에 대해 읽기 위해, 미디어위키 DOM 스펙을 참조하십시오.

Parsoid는 원래 웹 서비스로 구조화되었고 자바스크립트에서 작성되었으며, Node.js의 사용을 만듭니다. 2019년 2월 기술 토론 (발표 자료)과 블로그 게시물에서 포팅 과정을 설명합니다. Parsoid 확장 API는 현재 활발하게 개발 중입니다; 2020년 8월의 기술 토론은 이 작업을 설명합니다.

GitHub Repository: https://github.com/wikimedia/parsoid

Usage

Installation

미디어위키 1.35 LTS에서, Parsoid/PHP는 번들로 포함되고 시각 편집기에서 자동적으로 로딩됩니다. 단일 서버에서 사용되면 설정이 요구되지 않습니다.

만약 현재 개발 중인 미디어위키 1.36 버전을 사용하려면, 2020년 8월 24일부터 Parsoid는 명시적으로 로딩이 되어야 합니다 (자동-로딩 핵킹은 1.36-wmf.6에서 제거되었습니다). 다음을 LocalSettings.php에 추가하십시오: 

wfLoadExtension( 'Parsoid', __DIR__ . 'vendor/wikimedia/parsoid/extension.json' );

이것은 1.36의 출시에 대해 변경될 것으로 예상됩니다.

Development

Development happens in the Parsoid Git repository. Code review happens in Gerrit. See Gerrit/Getting started to set up an account for yourself.

If you use the MediaWiki-Vagrant development environment using a virtual machine, you can simply add the role visualeditor to it and it will set up a working Parsoid along with Extension:VisualEditor. (This may have been broken by the switch to Parsoid/PHP: T258940)

Note that the most-recently released version of Parsoid is written in PHP, and installation of Parsoid/PHP is what is described below. This is what you should use if you are running MediaWiki 1.35 or later. Check Parsoid/JS if you are running the old version of Parsoid written in JavaScript, and used for MW 1.34 and earlier.

Linking a developer checkout of Parsoid

In a standard MediaWiki installation, Parsoid code is bundled in two different ways: first, Parsoid is included from MediaWiki as a composer library, wikimedia/parsoid. This contains the main codebase, but does not contain the REST API used by VisualEditor and RESTBase. In order to enable the REST API, the extension code included in the Parsoid library can be loaded with a call to wfLoadExtension(...) in your LocalSettings.php.

For development purposes you usually want to use a git checkout of Parsoid, and not the version bundled in MediaWiki core as a composer library. The following lines added to LocalSettings.php allow use of a git checkout of Parsoid (optionally), load the Parsoid REST API with wfLoadExtension (rather than using the version bundled in VisualEditor) and manually do the Parsoid configuration which is usually done by VisualEditor: 

$parsoidInstallDir = 'vendor/wikimedia/parsoid'; # bundled copy
#$parsoidInstallDir = '/my/path/to/git/checkout/of/Parsoid';

// For developers: ensure Parsoid is executed from $parsoidInstallDir,
// (not the version included in mediawiki-core by default)
// Must occur *before* wfLoadExtension()
if ( $parsoidInstallDir !== 'vendor/wikimedia/parsoid' ) {
    // AutoLoader::registerNamespaces was added in MW 1.39
    AutoLoader::registerNamespaces( [
        // Keep this in sync with the "autoload" clause in
        // $parsoidInstallDir/composer.json
        'Wikimedia\\Parsoid\\' => "$parsoidInstallDir/src",
    ] );
}

wfLoadExtension( 'Parsoid', __DIR__ . "$parsoidInstallDir/extension.json" );

# Manually configure Parsoid
$wgVisualEditorParsoidAutoConfig = false;
$wgParsoidSettings = [
    'useSelser' => true,
    'rtTestMode' => false,
    'linting' => false,
];
$wgVirtualRestConfig['modules']['parsoid'] = [
    // URL to the Parsoid instance.
    // If Parsoid is not running locally, you should change $wgServer to match the non-local host 
    // While using Docker in macOS, you may need to replace $wgServer with http://host.docker.internal:8080
    // While using Docker in linux, you may need to replace $wgServer with http://172.17.0.1:8080
    'url' => $wgServer . $wgScriptPath . '/rest.php',
    // Parsoid "domain", see below (optional, rarely needed)
    // 'domain' => 'localhost',
];
unset( $parsoidInstallDir );

These lines are not necessary for most users of VisualEditor, who can use auto-configuration and the bundled Parsoid code included in MediaWiki 1.35 and VisualEditor, but they will be required for most developers.

If you're serving MediaWiki with Nginx, you'll need to also add something like this in your server block (Assuming your MediaWiki setup has its files residing in /w/): 

location /w/rest.php/ {
    try_files $uri $uri/ /w/rest.php?$query_string;
}

To test proper configuration, visit {$wgScriptPath}/rest.php/{$domain}/v3/page/html/Main%20Page where $domain is the hostname in your $wgCanonicalServer. (Note that production WMF servers do not expose the Parsoid REST API to the external network.)

Running the tests

To run all parser tests and mocha tests: 

$ composer test

The parser tests have quite a few options now which can be listed using php bin/parserTests.php --help.

If you have the environment variable MW_INSTALL_DIR pointing to a configured MediaWiki installation, you can run some additional tests with: 

$ composer phan-integrated

Converting simple wikitext

You can convert simple wikitext snippets from the command line using the parse.php script in the bin/ directory: 

echo 'Foo' | php bin/parse.php

The parse script has a lot of options. php bin/parse.php --help gives you information about this.

Debugging Parsoid (for developers)

See Parsoid/Debugging for debugging tips.

Continuous Integration

As of October 2021

Parsoid is always available as a library since it is a composer dependency of MediaWiki core. But two pieces are not enabled:

  • Parsoid ServiceWiring
  • Parsoid's external REST api

The test runner Quibble would enable it if it detects mediawiki/services/parsoid.git has been cloned as part of the build. In which case it:

  • points the autoloader for Wikimedia\Parsoid to the cloned code (effectively replacing the version installed by composer)
  • Load the extension wfLoadExtension( 'Parsoid', '/path/to/cloned/repo' );

The ServiceWiring should be enabled in MediaWiki starting with 1.38.

The REST API would theorically never get merged in MediaWiki: a) it has never been exposed to the public in production, it is an internal API used by RESTBase which is going away; b) it never has been security audited and c) it is redundant with the enterprise MediaWiki API. The solution will be for VisualEditor to invoke Parsoid directly via the VisualEditor Action API which would save a round trip through the REST API.

Loading the extension is thus a hack which enables using interfaces subject to change and which we don't really want people to use yet.

For most purposes, parsoid should thus not be added as a CI dependency, the only exception as of October 2021 is the Disambiguator MediaWiki extension.

Loading parsoid as an extension let us run MediaWiki integration test jobs against mediawiki/services/parsoid.git (such as Quibble, apitesting) and ensure Parsoid and MediaWiki work together.

An extension may be able to write tests with Parsoid even when the repository has not been cloned. Since it is a composer dependency of MediaWiki core the MediaWiki\Parsoid namespace is available, but the service wiring part is not (it is extension/src in the Parsoid repository and exposed as the \MWParsoid namespace). The ParsoidTestFileSuite.php code would only run the parser tests if Parsoid has been loaded (which should be the default with MediaWiki 1.38).

For CI, Parsoid is tested against the tip of mediawiki, whereas mediawiki is tested with the composer dependency. In case of a breaking change, the Parsoid change get merged first (which breaks its CI but not MediaWiki one) and MediaWiki get adjusted when Parsoid is updated. It is thus a one way change.

Release build

For MediaWiki release builds, we have an integration of Parsoid ServiceWiring into VisualEditor in order to have VisualEditor work without further configuration (beside a wfLoadExtension( 'VisualEditor' )). The release build also enables the REST API and hook everything us so that parsoid works out of the box. This is done by copying a bit of parsoid code into VisualEditor which is not in the master branch of VisualEditor since that would be obsolete as soon as Parsoid is updated. Instead the code is maintained in two places.

Technical documents

See slso

Retrieved from "https://dawoum.duckdns.org/w/index.php?title=Parsoid&oldid=94575"