Sputnik

A Wiki in Lua

Yuri Takhteyev & Jim Whitehead II

July 14, 2008

Sputnik is a wiki

Sputnik is written in Lua

Sputnik is extensible ... into other things

Example 1: wowprogramming.com

Example 2: Tickets

Easy to install

$ bash kepler-install-1.1-1 --prefix=$SPUTNIK

Works with CGI, FastCGI, Xavante

Simple web API

#! /bin/bash /home/yuri/sputnik/bin/wsapi.cgi
require('sputnik')
local my_sputnik = sputnik.new{
   -- set a bunch of parameters
}
return function(wsapi_env)
   return my_sputnik.run(wsapi_env)
end

Or

#! /bin/bash /home/yuri/sputnik/bin/wsapi.cgi
require('sputnik')
return sputnik.new_wsapi_run_fn{
   -- set a bunch of parameters
}

Application side

local request = wsapi.request.new(wsapi_env)
request = self:translate_request(request)
local node = self:get_node(request.node_id)
local action_fn = self:get_action_fn(node, request.command)
local content, content_type = action_fn(node, request, self)    

local response = wsapi.response.new()
response.headers["Content-Type"] = content_type or "text/html"
response:write(content)
return response:finish()

Easy to install

$ ./bin/luarocks --from=$URL install sputnik

Easy to add plugins, libraries

$ ./bin/luarocks install wsapi-fcgi
$ ./bin/luarocks --from=$URL1 install sputnik-tickets
$ ./bin/luarocks --from=$URL2 install your-plugin

Core

Cosmo, Markdown

LuaFileSystem, MD5, lbase64, luasocket

Optional

LuaSQL, LuaSVN

Storage

Markup, templates, i18n

Object types

Actions

A simple API: "Versium"

Five implementations

• files & directories (131 lines)
• git (167 lines)
• mysql (208 lines)
• sqlite3 (200 lines)
• subversion (coming soon)

(See http://sputnik.freewisdom.org/en/Versium)

Markdown is the default

But anything is possible

• LuaPod (Sérgio Medeiros)
• Medialike (Hisham Muhammad)

1. Cosmo templates

$do_messages[[<p class="$class">$message</p>]]
<div class='content'>$content</div>

2. A Lua table

{
  do_messages      = node.messages,
  content          = node.inner_html,
}

Produces

<p class="notice">Successfully created your new account.\</p>
<div class='content'>\</div>

(See http://cosmo.luaforge.net)

1. Keys in templates

$if_logged_in[[ _(HI_USER) (<a $logout_link>_(LOGOUT)</a>) ]]

2. A translation file

HI_USER = {
   en_US = "Hi, $user!",
   ru    = "Превед, $user!",
   pt_BR = "Oi, $user!", 
}

3. A config value

INTERFACE_LANGUAGE = "ru"

Produces

Превед, Медвед!

A document-oriented hierarchical

storage system with history.

Chunks of data

Stored as Lua code

Stored with history (via "Versium")

Can be "activated"

Are self-describing

Use prototype inheritance

Can have children (sort of)

Request = node + command + parameters

• Lua_Workshop_2008 + .slides

Node + command ⇒ action function

• mapping can be overriden on per-node basis

Action function returns content + status

• HTML, XML, Lua code, a binary image

wowprogramming.com

Provide online reference material for book

Community discussion and support

Endpoint for targeted advertisement

Virtual Private Server with modest limits:

• 384MB memory (guaranteed)
• 768MB memory (burst)
• 15GB disk space
• 200GB bandwidth limit

Lighttpd web server

WSAPI/Kepler stack

Pre-release version of Earth (slightly customized)

Uses versium-mysql and sputnik-auth-mysql to speed up data access

Contains custom modules:

• API documentation
• Forums
• Node comments
• AJAX web-based Lua interpreter
• Node redirection

Databases

Creating a database schema for API documentation is relatively straightforward, but difficult to extend.

• Could make the schema extensible through use of tag/value pairs in a separate table
• Handles concurrent edits better than alternative methods due to centralized system/locking
• PROBLEM: How to create an intuitive web form for editing this data

XML

XML format is easy to validate and parse but difficult to write a schema for

• As new aspects are encountered, the schema or format must change
• Extremely difficult for a human to edit directly, so requires some sort of web form and validation.

Lua tables

• Lua tables are easy to serialize and read
• Adding new elements to API definitions is as simple as adding another key/value pair to the table.
• Easy to batch process
  • Simple text-based format allowing for storage in a text-based SCM for diff/history

API table for UnitHealth function

 arguments = {
  [1] = {
    name = "unit",
    desc = "The unit to query",
    type = "unitId",
  },
 }
 categories = "unit, stats"
 description = "Returns the current mana points of the given unit"
 returns = {
  [1] = {
    desc = "The unit's current mana points",
    name = "mana",
    type = "number",
  },
 }
 signature = [[mana = UnitMana("unit")]]

Although the ease of editing these tables is completely subjective, empirical observations show that these more concise definitions are easier to use than XML or direct entry into a database.

Use AJAX to check the syntax of the Lua definitions

Can provide error messages as well as a simple pass/fail

Perform more complex validation

• Only allow valid keys and value
• Check the content of description/signatures
• Possiblities are endless

600+ pages of World of Warcraft Programming: A Guide and Reference for Creating WoW Addons were created via two-step transformation:

• Generated markup from Lua definitions
• Run a macro to convert the markup into appropriate styling and formatting
• API documents on wowprogramming.com generated on-the-fly from Lua definitions

• Easy to edit
• Easy to parse
• Extensible

• Modular and easy to extend
• Versium and Saci naturally fit into the way we define API functions
• Active open source project, easy to contribute to

• WSAPI layer makes Kepler extremely easy to deploy
• Allows Sputnik to run as a WSAPI application via CGI/FastCGI
• Fits naturally into the application stack giving us a Lua solution for web deployment.

"Tickets" - a simple issue tracker

1. A prototype
2. Templates
3. Two actions

More Fields

fields= [[
  reported_by = {.11}
  priority    = {.12}
  component   = {.13}
  assigned_to = {.14}
  status      = {.15} 
  resolution  = {.16}
]]

Edit UI

NODE.edit_ui= [[
  reported_by    = {1.31, "text_field"}
  assigned_to    = {1.32, "text_field"}
  status         = {1.33, "select", 
                  options = {"open", "someday", ... }}
  resolution     = {1.35, "select",
                  options = {"n.a.", "fixed", "wontfix"}}
  priority       = {2.10, "select", 
                  options = {"unassigned", "high", ... }}
  page_name   = null
]]

actions= [[show = "tickets.show"]]

translations = "tickets/translations"
templates    = "tickets/templates"

<table width="100%">
 <tr style="background:$status_color">
  <td width="15%" style="text-align: right;">
   <span style="font-size: 80%">ticket id</span><br/>
   <span style="font-size: 200%;">$ticket_id</span>
  </td>
  <td width="15%" style="text-align: right;">
   <span style="font-size: 80%">status</span><br/>
   <span style="font-size: 200%">$status</span>
   ...

                       $status_color  


                                  $ticket_id



                                 $status

actions.show = function(node, request, sputnik)
   local parent_id = node.id:match(PARENT_PATTERN)
   local index_node = sputnik:get_node(parent_id)
   local ticket_info = {
      status       = node.status
      status_color = status_to_color[node.status] or "white",
      ticket_id    = node.id:gsub(parent_id.."/", ""),
      ...
   }
   ...
   node.inner_html = cosmo.fill(node.templates.SHOW, ticket_info)
   return node.wrappers.default(node, request, sputnik)
end

http://sputnik.freewisdom.org/

http://sputnik.freewisdom.org/en/Lua_Workshop_2008.slides