Wikipedia:Wikipedia Signpost/2012-08-20/Technology report

Lua onto test2wiki and news of a convention-al extension: New embeddable scripting ("template replacement") language Lua received considerable scrutiny this week when it began its long road to widespread deployment, landing on the test2wiki test site on Wednesday (wikitech-l mailing list). ... the fourth in our series profiling participants in this year's Google Summer of Code (GSoC) programme.

Technology report

Lua onto test2wiki and news of a convention-al extension

By Jarry1250

Lua starts first high-profile testing phase

New embeddable scripting ("template replacement") language Lua received considerable scrutiny this week when it began its long road to widespread deployment, landing on the test2wiki test site on Wednesday (wikitech-l mailing list).

Specifically, under the direction of WMF lead platform architect Tim Starling, two extensions were deployed to test2wiki ahead of a deployment to MediaWiki.org in the near future: Extension:Scribunto, which acts as an interface between wikitext and a backend Lua interpreter, and Extension:CodeEditor, an extension that drastically improves the edit page for Lua modules.

In terms of design, several things have changed since Lua was first mentioned in the Signpost back in January this year, but the thrust is similar. If this first deployment is anything to go by, Lua integration will mean the creation a new module: namespace in which to host Lua scripts and the introduction of an {{#invoke:...|}} parser function. Communities will be expected to use only #invoke within template space in much the same way as they currently use other parser functions such as #if.

The gains are both potentially very significant – faster template load times, plus cleaner and more powerful template code – and are largely undisputed. Talks explaining Lua were well-received at both Berlin and Washington. The only criticism from developers was that Lua, while a step in the right direction, is not the perfect solution: Can Wikimedians really be expected to learn a whole new programming language? Should there not be a central repository of Lua scripts? Might Lua not be too simple to meet wikis' ever expanding templating requirements? For now, however, developers await with cautious optimism.

Google Summer of Code: the Convention extension

For the fourth in our series profiling participants in this year's Google Summer of Code (GSoC) programme, in which student developers are paid to contribute code to MediaWiki, the Signpost caught up with Akshay Chugh, a recent electronics and instrumentation graduate working out of the Indian city of Jaipur. Originally fascinated by user interface design, Akshay Chugh has more recently turned his attention to designing an extension that can turn a vanilla MediaWiki installation into one immediately suitable for use as a "convention" (for example, Wikimania) hub.

“

A screenshot of the "conference dashboard", the main entry point for users of the Convention extension

From the moment I chose this project (now known as ConventionExtension), I knew that when it came to the many features that this could incorporate, the list was endless. So for this span of three months I wanted to select the features that would be most crucial and that were most commonly found in other conference management systems. At the same time, I wanted to end up with a well-built (extensible and flexible) architecture. This extension would interact with two sets of users, namely the admin group (administrators of the wiki) and the default users group (people registering for the conference). So the idea was simple: make the process of an admin setting up a conference on a wiki as automated as possible and the process of user interactions with the system as satisfying as possible in terms of the easy flow between the various pages.

As of now, the dashboard page (which is where an admin will spend all of their time while interacting with this extension) works as it should, and other essential functionality, including the author registration page, is good to go. At this point it still lacks some other features such as an account setup page, internationalisation/localisation support, and a payment gateway which will probably give this whole setup a lot more meaning. But overall, I would say I'm pretty content with the end result. Needless to say, some of the things I'd thought would be pretty straightforward and easy to implement didn't turn out to be. In particular, one of my goals with this extension was to make it as generic as possible to allow its users to customize it in whichever way they want. Hence the difficulty: every decision made and every step taken forward had to be done with a sense of being extensible and customizable.

During the course of this project I haven't implemented many features with performance in mind, having said that I presume many of the difficulties and mind-boggling issues are yet to come in the weeks after GSoC.

”

In brief

Signpost poll

translatewiki.net

You can now give your opinion on next week's poll: Would you learn to program templates in Lua?

Not all fixes may have gone live to WMF sites at the time of writing; some may not be scheduled to go live for several weeks.

• MediaWiki 1.20wmf10 begins deployment cycle: 1.20wmf10 – the tenth release to Wikimedia wikis from the 1.20 branch – was deployed to its first wikis on August 20 and will be deployed to all wikis by August 29. The release incorporates about 300 changes to the MediaWiki software that powers Wikipedia, comprising 146 "core" changes plus a similar number of patches for WMF-deployed extensions. Among the changes (the product of some 14 days of development time) are fixes for bugs #33037 ("Special:Newfiles treats its subpage parameter as a limit") and #12701 ("Use diff of all unseen revisions in the new messages bar"), plus enhancements for Special:Mostlinkedcategories and syntax highlighting.
• Signpost app undergoes first release candidate: The first release candidate for a new Signpost Android app was released this week by its developers, Yuvi Panda and Shankar Narayan. Although Panda is a former WMF mobile team member, both he and Narayan are developing the free-to-download and open-source app in their spare time; it already features a more comfortable interface than one would get accessing traditional Signpost pages from a mobile browser, and the ability to link up with any Facebook or Twitter apps loaded onto the device; it will also feature notifications of new issues, Panda reports; Panda will put out requests for possible testers to come forward (mobile-l mailing list). Similar positive news came this week from the team working on the Wiki Loves Monuments mobile app, which released the second beta from their 1.1 series (wikitech-l).
• Thread deletions prompt Mailman turmoil: Mailing-list software GNU Mailman's inability to handle anything other than sequentially numbered emails came to the fore this week after hundreds of links to previous messages appeared to break overnight, possibly with the added complication of archive corruption (wikitech-l). The problem was soon tracked to a series of message deletions undertaken on request by the WMF operations team. Despite efforts to undo the effect of the deletions on Mailman's numerically based index, as of time of writing, many links to /pipermail/-form URLs remain broken. The frequency and severity of such instances occurring in past months and years is unknown (recent editions of the "Technology report" are unaffected due to their use of an external mail archive system to provide permanent links).
• Mailman turmoil prompts code of conduct suggestion: The same mailing-list thread as with regards mailing-list archives also addressed the issue of the correct tone and style to adopt during public email exchanges. The discussion came after comments by wikitech-l veteran MZMcBride to WMF operations engineer Daniel Zahn (the staff member responsible for the mailing-list deletions) were picked out by other list members as being needlessly harsh and personalised. The response to the incident was mixed, with staff and volunteer developers airing a wide range of views, including the suggestion of a "code of conduct" for the mailing list; this suggestion has garnered significant support at the time of writing.
• EtherEditor undergoes stress test: Since the rise to prominence of collaborative note-taking system Etherpad, the allure of real-time collaborative editing with Wikimedia wikis has led several developers to try their hand at integrating its functionality into MediaWiki. Although not nearly production-ready, the stress-testing of WMF contractor Mark Traceur's EtherEditor passed off mostly successfully this week (wikitech-l), making it perhaps the most developed option on the table for MediaWiki system administrators.

← Previous "Technology report"

Next "Technology report" →

In this issue

20 August 2012

Op-ed

News and notes

In the news

Featured content

Technology report

WikiProject report

+ Add a comment

Discuss this story

These comments are automatically transcluded from this article's talk page. To follow comments, add the page to your watchlist. If your comment has not appeared here, you can try .

• Please provide a link to the on-wiki discussion about introducing Lua for end-user/contributor programming of Wikipedia. I tried a quick search of the Wikipedia namespace but didn't find it. Thanks. ~ Ningauble (talk) 19:24, 21 August 2012 (UTC)
  • There's been (to my knowledge) no community-specific discussion of Lua (except perhaps on MediaWiki.org), meaning that individual projects might still reject Lua as a programming language for their wikis, though there's no indication that theey would want to. For the discussion which resulted in WMF staff developers working on a Lua proposal rather than a different language, see our previous report. - Jarry1250 ^{[Deliberation needed]} 12:10, 22 August 2012 (UTC)
    • Ok. It was sort of a rhetorical question, with this week's Op-Ed piece in mind. (Considering that the complexity of templates is often cited as an impediment to ease of contribution, I wondered whether replacing (or augmenting) a relatively simple (in principle) macro substitution scheme with a full-blown Turing-complete programming language is really something that a lot of ordinary contributors have been begging for.) ~ Ningauble (talk) 15:29, 22 August 2012 (UTC)
      • It's a fair point. However, I don't think anyone's suggesting removing existing functionality, just adding new functionality for power templates (which only a small fraction of editors go near anyways). - Jarry1250 ^{[Deliberation needed]} 15:40, 22 August 2012 (UTC)
        • Ok, I guess this is not the place to discuss the merits of addressing the usability problem caused by increasingly complex templates by means of introducing a tool that even fewer people will be competent to use. (It is not only template writers who are impacted, but also template users who must grapple with the complexity of their creations.) We will see what develops – I might even use it myself, but I have years of experience developing software tools with a view toward usability. Thanks for covering the news. ~ Ningauble (talk) 21:10, 22 August 2012 (UTC)
  • Probably the reason you've been unable to find it is because the decision was made so long ago. On-wiki discussions of the general need for a better template programming language have been occurring since about 2006: people have been remarkably creative in turning our current template syntax into a Turing-complete programming language, first with pure-wiki versions (which often caused performance problems), then with ParserFunctions (which really demonstrated the limitations of the template syntax for programming). I'm not aware of any on-Enwiki decision specifically about using Lua as that better language, but the decision to have a better language was made about six years ago and this is simply the long-delayed implementation. --Carnildo (talk) 21:03, 22 August 2012 (UTC)
    • Power users always want more power, so I have no doubt that some people have been agitating for something like this all along. When I see highly complex templates, I often wonder whether people are doing things the wrong way because the tool is cumbersome, or whether they are just trying to do the wrong thing. I am sure there is a mix of both. It will be interesting to see what happens when more power becomes available, and we see whether 'more is better'. ~ Ningauble (talk) 21:29, 22 August 2012 (UTC)
• Why does all the important stuff take place off Wikipedia. It's impossible to find anything on Meta or those other sites of WMF. Really leaves out the Wikipedia community. MathewTownsend (talk) 00:44, 22 August 2012 (UTC)
  • Would you rather it be on en.wp (ie here?) Ed ^{[talk] [majestic titan]} 06:45, 22 August 2012 (UTC)
  • You mean the mailing list discussions? - Jarry1250 ^{[Deliberation needed]} 12:10, 22 August 2012 (UTC)
    • Personally, I (in both a personal and professional capacity) always regret that we rely on mailing lists so much. It's a very outdated format without any filtering (If you're signed up to wikitech-l, you get all of wikitech-l, not just the discussions you're interested in), with odd conventions, and with relatively little transparency for on-wiki people. I've actually made it a principle recently to exclusively announce things on-wiki; you hit a much more representative group of users (and a much larger group of users) and people passing by in the course of their editing have an opportunity to join the discussion. However, there are issues with handling things on Enwiki and enwiki only.
    • We've got more than 200 projects, in a pile of languages; merely in English alone we've got Wikiquote, Wikisource, Wikinews, so on, so forth. While Enwiki is our flagship project in terms of reader numbers, editor numbers, content and complexity, we cannot exclude other projects and their concerns and needs from the conversation. Holding things exclusively on enwiki excludes people from other projects from the conversation. Having said that, holding discussions on mailing lists tends to do that pretty well too, and the same would be true if we held discussions on wikiversity or fr.wiki or Meta or, well, anywhere else.
    • Ultimately it comes down to two elements; ease of browsing and language barriers. Ease of browsing is the simple fact that if I start a discussion on meta, people from enwiki or fr.wiki don't necessarily get to see it - unless I send out individualised notifications, it's hard to notify people because a lot of Wikimedians simply don't browse meta and so announcements at their community venues go unheard by the wider movement. Even if people do stumble across the notifications or I do send people individualised pokes on their "home" projects, meta is not a place where they feel at home (you show up with a redlinked userpage, any of your custom preference settings gone and no tiein to the community there) and it's very hard to actually follow a discussion (each project has their own watchlists; if I'm an enwiki user who goes to an enwiki discussion, I watchlist it and get updates in the course of editing. If it's on metawiki, it doesn't appear in the watchlist I actually pay attention to).
    • Language barriers: what it says on the tin. People write our content in a heck of a lot of languages, which is fantastic :). It does, however, make keeping people in the loop difficult. By hosting discussions on enwiki or even meta, I'm normally hosting them in English - and if someone only speaks French or Dutch or Hindi, they'll find it very difficult to participate (if I can even find the translation resources to let them know the conversation is going on). I'm genuinely not sure how to solve the language barriers problem elegantly, but we need to get better at it - the barriers to browsing will hopefully be reduced by the introduction of globalised user profiles and a full, interwiki notifications system like mw:Echo. Okeyes (WMF) (talk) 07:49, 27 August 2012 (UTC)

Tutorial?

Is there a getting started using Lua on Wikipedia tutorial somewhere? Klortho (talk) 01:10, 28 August 2012 (UTC)

• Tim Starling's Wikimania talk was published this week and is conveniently linked from Wikipedia:Wikipedia Signpost/2012-08-27/Technology report. It's not really a tutorial, but it's certainly a very good start. - Jarry1250 ^{[Deliberation needed]} 09:18, 28 August 2012 (UTC)

The Signpost is written by editors like you — join in!

Home

About

Archives

Newsroom

Subscribe

Suggestions