A list member who wanted to stay unnamed pointed out that Sputnik's
performance can be improved quite a bit by telling the client to cache
javascript and css files. I added support for this, and also made some
other changes to make YSlow happy.

1. Caching

Nodes have two new fields: http_cache_control and http_expires. Both
are set to nil at the level of @Root, but are then re-set for @CSS,
@JavaScript, @Binary_File and sputnik/css/colors - for now to
"max-age=3D3600" and "2" respectively.  The value of http_cache_control
is passed directly to the client as "Cache-Control" header, and
http_expires is interpreted as an offset (in hours) to set the
timestamp in the "Expires" header.  (I expect to tweak the value and
am looking for suggestions.)

This means that if you do nothing then images, css files and
javascript files will be cached for an hour by the client. If you are
making changes to those nodes (e.g., uploading a new logo) and want to
see the effects, you will need to do click on "reload" button in the
browser while holding shift. (Your users won't see the new logo until
their cache expires.) This seems to speed up the loading by a factor
of 3, after the first page view.

2. JavaScript at the bottom.

I moved all JavaScript to the bottom of the page, as YSlow recommends.
This doesn't seem to make any drastic changes, though I didn't measure
it very precisely.

3. Gzipping content

I used Igancio Burge=F1o's code (see an email to the Kepler list in
November) to add support for content compression. To enable this you
would need to set "USE_COMPRESSION" config parameter to "true" and
install lzlib rock. (Available from the main repository: "luarocks
install lzlib".) Again, I haven't noticed any huge improvements from
where I am at the moment, but perhaps in other cases it would make
more of a difference.

YSlow seems to be quite happy now, only wishing that we were using a
content delivery network. I am guessing further improvements in
performance would need to be from server-side profiling or from
server-side caching. (Or from adding support for content delivery
networks, as YSlow suggests.)

 - yuri

--=20
http://sputnik.freewisdom.org/

_______________________________________________
Sputnik-list mailing list
Sputnik-list@lists.luaforge.net
http://lists.luaforge.net/cgi-bin/mailman/listinfo/sputnik-list

On Jan 11, 2009, at 6:08 PM, Yuri Takhteyev wrote:

> A list member who wanted to stay unnamed pointed out that Sputnik's
> performance can be improved quite a bit by telling the client to cache
> javascript and css files. I added support for this, and also made some
> other changes to make YSlow happy.

You might also want to consider e-tag and conditional requests as well  
(e.g. If-None-Match & 304 Not Modified).

That said... considering that Sputnik usually runs in a full fledge  
server such as Apache... why bother re-implementing all those features  
in Sputnik itself?

Cheers,

--
PA.
http://alt.textdrive.com/nanoki/

_______________________________________________
Sputnik-list mailing list
Sputnik-list@lists.luaforge.net
http://lists.luaforge.net/cgi-bin/mailman/listinfo/sputnik-list

On Jan 11, 2009, at 6:08 PM, Yuri Takhteyev wrote:

> A list member who wanted to stay unnamed pointed out that Sputnik's
> performance can be improved quite a bit

For what it's worth, here are some basic numbers from ApacheBench:

Apache -> CGI -> Sputnik

% ab -k -n 100 http://sputnik.freewisdom.org/
Document Length:        15118 bytes
Requests per second:    0.78 [#/sec] (mean)
Time per request:       1274.200 [ms] (mean)

For comparison, here are the Nanoki numbers:

Nginx -> HTTP -> Nanoki

% ab -n 100 http://svr225.stepx.com:3388/drama
Document Length:        22601 bytes
Requests per second:    2.09 [#/sec] (mean)
Time per request:       477.961 [ms] (mean)

Not sure why the one to two difference here (0.7 vs. 2.09). Perhaps  
the CGI overhead on the Sputnik side, versus plain HTTP on the Nanoki  
side?


One way or another, using a persistent connection improve things a bit:

% ab -k -n 100 http://svr225.stepx.com:3388/drama
Document Length:        22601 bytes
Requests per second:    7.15 [#/sec] (mean)
Time per request:       139.844 [ms] (mean)

Unfortunately, the Apache instance fronting Sputnik is not set to  
handle persistent connection, so no gain there.

HTH.

Cheers,

--
PA.
http://alt.textdrive.com/nanoki/


_______________________________________________
Sputnik-list mailing list
Sputnik-list@lists.luaforge.net
http://lists.luaforge.net/cgi-bin/mailman/listinfo/sputnik-list

On Thu, Jan 15, 2009 at 9:57 PM, Petite Abeille
<petite.abeille@gmail.com> wrote:
>
> On Jan 11, 2009, at 6:08 PM, Yuri Takhteyev wrote:
>
>> A list member who wanted to stay unnamed pointed out that Sputnik's
>> performance can be improved quite a bit
>
> For what it's worth, here are some basic numbers from ApacheBench:
>
> Apache -> CGI -> Sputnik
>
> % ab -k -n 100 http://sputnik.freewisdom.org/
> Document Length:        15118 bytes
> Requests per second:    0.78 [#/sec] (mean)
> Time per request:       1274.200 [ms] (mean)

Some comparisons against a real production site that is running
Sputnik.  The main Sputnik site does not take advantage of many of the
rocks that are available when running a real site:

% ab -n 100 http://wowprogramming.com/
HTML transferred:       343800 bytes
Requests per second:    3.10 [#/sec] (mean)
Time per request:       322.890 [ms] (mean)

Keep in mind my site isn't as speedy as it could be, and it certainly
suffers on the more dynamic pages (the following numbers are without
any query or node caching):

% ab -k -n 100 http://wowprogramming.com/forums/development
HTML transferred:       2669900 bytes
Requests per second:    0.48 [#/sec] (mean)
Time per request:       2066.132 [ms] (mean)

> For comparison, here are the Nanoki numbers:
>
> Nginx -> HTTP -> Nanoki
>
> % ab -n 100 http://svr225.stepx.com:3388/drama
> Document Length:        22601 bytes
> Requests per second:    2.09 [#/sec] (mean)
> Time per request:       477.961 [ms] (mean)
>
> Not sure why the one to two difference here (0.7 vs. 2.09). Perhaps the CGI
> overhead on the Sputnik side, versus plain HTTP on the Nanoki side?

There is a massive complexity difference between Nanoki and Sputnik
which is partially shown in the above benchmarks.
http://wowprogramming.com/ is able to do the following using Sputnik:

* Serve forums for discussion (rudimentary, but they work)
* Generate API documentation and listings dynamically from Lua tables
* Act as an AJAX Lua interpreter providing a Lua command-line on the web
* Convert .BMP format images to .PNG

and the list continues on.  The conceptual base of Sputnik is a high
feature 'node' that includes full inheritance.  You pay with a bit of
complexity for flexibility in implementation.  I would make the
statement that although Nanoki and Sputnik have both been called a
"wiki", the latter is a wiki only in how it's used in its default
state.

> One way or another, using a persistent connection improve things a bit:
>
> % ab -k -n 100 http://svr225.stepx.com:3388/drama
> Document Length:        22601 bytes
> Requests per second:    7.15 [#/sec] (mean)
> Time per request:       139.844 [ms] (mean)
>
> Unfortunately, the Apache instance fronting Sputnik is not set to handle
> persistent connection, so no gain there.

Here are all of the numbers, for comparison.  You can see that Nanoki
with keep-alive wins by quite a bit which is to be expected.

=======================================
% ab -n 100 http://wowprogramming.com/
HTML transferred:       343800 bytes
Requests per second:    3.08 [#/sec] (mean)
Time per request:       324.296 [ms] (mean)
=======================================
% ab -k -n 100 http://wowprogramming.com/
HTML transferred:       343800 bytes
Requests per second:    4.68 [#/sec] (mean)
Time per request:       213.586 [ms] (mean)
=======================================
% ab -n 100 http://svr225.stepx.com:3388/drama
HTML transferred:       2260100 bytes
Requests per second:    3.39 [#/sec] (mean)
Time per request:       294.658 [ms] (mean)
=======================================
% ab -k -n 100 http://svr225.stepx.com:3388/drama
HTML transferred:       2260100 bytes
Requests per second:    11.64 [#/sec] (mean)
Time per request:       85.921 [ms] (mean)
=======================================

- Jim

_______________________________________________
Sputnik-list mailing list
Sputnik-list@lists.luaforge.net
http://lists.luaforge.net/cgi-bin/mailman/listinfo/sputnik-list

On Jan 15, 2009, at 11:29 PM, Jim Whitehead II wrote:

> There is a massive complexity difference between Nanoki and Sputnik  
> which is partially shown in the above benchmarks.... The conceptual  
> base of Sputnik is a high feature 'node' that includes full  
> inheritance.  You pay with a bit of complexity for flexibility in  
> implementation.  I would make the statement that although Nanoki and  
> Sputnik have both been called a "wiki", the latter is a wiki only in  
> how it's used in its default state.

Fair enough... that said, my point was more related to Yuri's original  
concern... high level or not, Sputnik, in its wiki incarnation or  
otherwise, can directly benefits from a couple of rather  
straightforward tweaking in how it's setup in the overall HTTP food  
chain...

Cheers,

--
PA.
http://alt.textdrive.com/nanoki/ 

_______________________________________________
Sputnik-list mailing list
Sputnik-list@lists.luaforge.net
http://lists.luaforge.net/cgi-bin/mailman/listinfo/sputnik-list

On Thu, Jan 15, 2009 at 11:07 PM, Petite Abeille
<petite.abeille@gmail.com> wrote:
>
> On Jan 15, 2009, at 11:29 PM, Jim Whitehead II wrote:
>
>> There is a massive complexity difference between Nanoki and Sputnik which
>> is partially shown in the above benchmarks.... The conceptual base of
>> Sputnik is a high feature 'node' that includes full inheritance.  You pay
>> with a bit of complexity for flexibility in implementation.  I would make
>> the statement that although Nanoki and Sputnik have both been called a
>> "wiki", the latter is a wiki only in how it's used in its default state.
>
> Fair enough... that said, my point was more related to Yuri's original
> concern... high level or not, Sputnik, in its wiki incarnation or otherwise,
> can directly benefits from a couple of rather straightforward tweaking in
> how it's setup in the overall HTTP food chain...

Definitely, I just wanted to show that wowprogramming.com is also
running Sputnik but as you can see it's MUCH closer to the numbers (at
least without keepalive) that Nanoki and push out.  I don't do
anything special, just use a few of the more advanced rocks based on
my server setup and architecture.

- Jim

_______________________________________________
Sputnik-list mailing list
Sputnik-list@lists.luaforge.net
http://lists.luaforge.net/cgi-bin/mailman/listinfo/sputnik-list

--===============0817566325==
Content-Type: multipart/alternative; 
	boundary="----=_Part_5652_21048882.1232061407203"

------=_Part_5652_21048882.1232061407203
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

FWIW, this is from luarocks.org, running on the cheapest slice in Slicehost,
with Xavante:

Requests per second:    1.91 [#/sec] (mean)
Time per request:       524.747 [ms] (mean)

Again, no caching whatsoever, with some node caching I believe these numbers
could be easily least 10x more. Maybe some kind of caching for wiki-nodes
(with invalidation on edits) could be implemented? I have rudimentary
caching support on Orbit (which I use on http://www.lua.inf.puc-rio.br), and
it is getting 21 req/s on a NSLU2 :-).

--
Fabio

On Thu, Jan 15, 2009 at 9:10 PM, Jim Whitehead II <jnwhiteh@gmail.com>wrote:

> On Thu, Jan 15, 2009 at 11:07 PM, Petite Abeille
> <petite.abeille@gmail.com> wrote:
> >
> > On Jan 15, 2009, at 11:29 PM, Jim Whitehead II wrote:
> >
> >> There is a massive complexity difference between Nanoki and Sputnik
> which
> >> is partially shown in the above benchmarks.... The conceptual base of
> >> Sputnik is a high feature 'node' that includes full inheritance.  You
> pay
> >> with a bit of complexity for flexibility in implementation.  I would
> make
> >> the statement that although Nanoki and Sputnik have both been called a
> >> "wiki", the latter is a wiki only in how it's used in its default state.
> >
> > Fair enough... that said, my point was more related to Yuri's original
> > concern... high level or not, Sputnik, in its wiki incarnation or
> otherwise,
> > can directly benefits from a couple of rather straightforward tweaking in
> > how it's setup in the overall HTTP food chain...
>
> Definitely, I just wanted to show that wowprogramming.com is also
> running Sputnik but as you can see it's MUCH closer to the numbers (at
> least without keepalive) that Nanoki and push out.  I don't do
> anything special, just use a few of the more advanced rocks based on
> my server setup and architecture.
>
> - Jim
>
> _______________________________________________
> Sputnik-list mailing list
> Sputnik-list@lists.luaforge.net
> http://lists.luaforge.net/cgi-bin/mailman/listinfo/sputnik-list
>

------=_Part_5652_21048882.1232061407203
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

FWIW, this is from <a href="http://luarocks.org">luarocks.org</a>, running on the cheapest slice in Slicehost, with Xavante:<br><br>Requests per second:&nbsp;&nbsp;&nbsp; 1.91 [#/sec] (mean)<br>Time per request:&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 524.747 [ms] (mean)<br>
<br>Again, no caching whatsoever, with some node caching I believe these numbers could be easily least 10x more. Maybe some kind of caching for wiki-nodes (with invalidation on edits) could be implemented? I have rudimentary caching support on Orbit (which I use on <a href="http://www.lua.inf.puc-rio.br">http://www.lua.inf.puc-rio.br</a>), and it is getting 21 req/s on a NSLU2 :-).<br>
<br clear="all">--<br>Fabio<br><br><div class="gmail_quote">On Thu, Jan 15, 2009 at 9:10 PM, Jim Whitehead II <span dir="ltr">&lt;<a href="mailto:jnwhiteh@gmail.com">jnwhiteh@gmail.com</a>&gt;</span> wrote:<br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
On Thu, Jan 15, 2009 at 11:07 PM, Petite Abeille<br>
<div class="Ih2E3d">&lt;<a href="mailto:petite.abeille@gmail.com">petite.abeille@gmail.com</a>&gt; wrote:<br>
&gt;<br>
</div><div class="Ih2E3d">&gt; On Jan 15, 2009, at 11:29 PM, Jim Whitehead II wrote:<br>
&gt;<br>
&gt;&gt; There is a massive complexity difference between Nanoki and Sputnik which<br>
&gt;&gt; is partially shown in the above benchmarks.... The conceptual base of<br>
&gt;&gt; Sputnik is a high feature &#39;node&#39; that includes full inheritance. &nbsp;You pay<br>
&gt;&gt; with a bit of complexity for flexibility in implementation. &nbsp;I would make<br>
&gt;&gt; the statement that although Nanoki and Sputnik have both been called a<br>
&gt;&gt; &quot;wiki&quot;, the latter is a wiki only in how it&#39;s used in its default state.<br>
&gt;<br>
&gt; Fair enough... that said, my point was more related to Yuri&#39;s original<br>
&gt; concern... high level or not, Sputnik, in its wiki incarnation or otherwise,<br>
&gt; can directly benefits from a couple of rather straightforward tweaking in<br>
&gt; how it&#39;s setup in the overall HTTP food chain...<br>
<br>
</div>Definitely, I just wanted to show that <a href="http://wowprogramming.com" target="_blank">wowprogramming.com</a> is also<br>
running Sputnik but as you can see it&#39;s MUCH closer to the numbers (at<br>
least without keepalive) that Nanoki and push out. &nbsp;I don&#39;t do<br>
anything special, just use a few of the more advanced rocks based on<br>
my server setup and architecture.<br>
<font color="#888888"><br>
- Jim<br>
</font><div><div></div><div class="Wj3C7c"><br>
_______________________________________________<br>
Sputnik-list mailing list<br>
<a href="mailto:Sputnik-list@lists.luaforge.net">Sputnik-list@lists.luaforge.net</a><br>
<a href="http://lists.luaforge.net/cgi-bin/mailman/listinfo/sputnik-list" target="_blank">http://lists.luaforge.net/cgi-bin/mailman/listinfo/sputnik-list</a><br>
</div></div></blockquote></div><br>

------=_Part_5652_21048882.1232061407203--


--===============0817566325==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
Sputnik-list mailing list
Sputnik-list@lists.luaforge.net
http://lists.luaforge.net/cgi-bin/mailman/listinfo/sputnik-list

--===============0817566325==--

> Not sure why the one to two difference here (0.7 vs. 2.09). Perhaps the CGI
> overhead on the Sputnik side, versus plain HTTP on the Nanoki side?

Perhaps that, or perhaps one computer is simply slower than the
other... Here is another data point: tnik.org is running identical
software on a different server, and guess what...

ab -n 100 http://tnik.org/
Requests per second:    3.73 [#/sec] (mean)
Time per request:       268.439 [ms] (mean)

That's 6 times faster than sputnik.freewisdom.org and twice as fast as
nanoki on stepx. The point being: benchmarking software running on
different servers just doesn't make much sense.

Not to mention that comparing CGI performance to that of a long
running process just seems silly. sputnik.freewisdom.org is running on
a server where I cannot run another long running process. Good luck
installing nanoki in this situation. :)

- yuri

-- 
http://sputnik.freewisdom.org/

_______________________________________________
Sputnik-list mailing list
Sputnik-list@lists.luaforge.net
http://lists.luaforge.net/cgi-bin/mailman/listinfo/sputnik-list

On Jan 16, 2009, at 3:32 AM, Yuri Takhteyev wrote:

>  is running on a server where I cannot run another long running  
> process. Good luck installing nanoki in this situation. :)

Nanoki can run as a short lived process as well.

For example, using a tcp wrapper such as DJB's tcpserver[1] or  
similar, Nanoki can be executed as a short lived script to handle one  
incoming TCP connection:

% tcpserver 127.0.0.1 1080 lua Nanoki.lua .

Cheers,

--
PA.
http://alt.textdrive.com/nanoki/

[1] http://cr.yp.to/ucspi-tcp/tcpserver.html

_______________________________________________
Sputnik-list mailing list
Sputnik-list@lists.luaforge.net
http://lists.luaforge.net/cgi-bin/mailman/listinfo/sputnik-list