--===============1017179364==
Content-Type: multipart/alternative; 
	boundary="----=_Part_20039_19146041.1232377698251"

------=_Part_20039_19146041.1232377698251
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline

Hi, list!

I was wondering this weekend while trying to import my WikiMedia data to
Sputnik:
Is there a way of creating a node that kind of 'filters' another node,
passed to him as a parameter?

I am thinking something in the line of:

http://spoo.tnik.org/en/Tags:technology
http://spoo.tnik.org/en/Category:technology

This could trigger a standard action (like a Category.filter action
receiving the node 'technology')

Is there some (non-hacking) way to do that?

Lu=EDs Eduardo Jason Santos

------=_Part_20039_19146041.1232377698251
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline

Hi, list!<br><br>I was wondering this weekend while trying to import my Wik=
iMedia data to Sputnik:<br>Is there a way of creating a node that kind of &=
#39;filters&#39; another node, passed to him as a parameter?<br><br>I am th=
inking something in the line of:<br>
<br><a href=3D"http://spoo.tnik.org/en/Tags:technology">http://spoo.tnik.or=
g/en/Tags:technology</a><br><a href=3D"http://spoo.tnik.org/en/Category:tec=
hnology">http://spoo.tnik.org/en/Category:technology</a><br><br>This could =
trigger a standard action (like a Category.filter action receiving the node=
 &#39;technology&#39;)<br>
<br>Is there some (non-hacking) way to do that?<br><br clear=3D"all">Lu=EDs=
 Eduardo Jason Santos<br>

------=_Part_20039_19146041.1232377698251--


--===============1017179364==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
Sputnik-list mailing list
Sputnik-list@lists.luaforge.net
http://lists.luaforge.net/cgi-bin/mailman/listinfo/sputnik-list

--===============1017179364==--

On Mon, Jan 19, 2009 at 3:08 PM, Lu=EDs Eduardo Jason Santos
<jasonsantos@gmail.com> wrote:
> Hi, list!
>
> I was wondering this weekend while trying to import my WikiMedia data to
> Sputnik:
> Is there a way of creating a node that kind of 'filters' another node,
> passed to him as a parameter?
>
> I am thinking something in the line of:
>
> http://spoo.tnik.org/en/Tags:technology
> http://spoo.tnik.org/en/Category:technology
>
> This could trigger a standard action (like a Category.filter action
> receiving the node 'technology')
>
> Is there some (non-hacking) way to do that?
>
> Lu=EDs Eduardo Jason Santos

That's an interesting question.  The only way I can think of
accomplishing this in the current version of Sputnik would be to use
prefix patterns or child defaults and create the URLs like so:

http://spoo.tnik.org/en/Tags/technology
http://spoo.tnik.org/en/Category/technology

I'm not sure if child defaults can handle patterns (I'd have to look)
but if so you'd just set it so that any child node of /Tags is set to
use a custom action for viewing that handles scanning the valid nodes
in your wiki and displaying those with the correct tags (or
categories).

If you are using non-clean URLs you can include a new GET parameter
without making things any nastier and you can access that directly
through the request in your custom action.

I'm interested in the concept of being able to pass parameters to a
given node rather than always having to access a sub-node.  That could
open up a lot of really nice applications that right now aren't
terribly easy to accomplish.

- Jim

_______________________________________________
Sputnik-list mailing list
Sputnik-list@lists.luaforge.net
http://lists.luaforge.net/cgi-bin/mailman/listinfo/sputnik-list

On Mon, Jan 19, 2009 at 1:19 PM, Jim Whitehead II <jnwhiteh@gmail.com> wrot=
e:
> ...
> I'm interested in the concept of being able to pass parameters to a
> given node rather than always having to access a sub-node.  That could
> open up a lot of really nice applications that right now aren't
> terribly easy to accomplish.

I also would like having such kind of "path processing". OTOH, this
may impose a certain load on the dispatcher (I'm not sure how it is
working currently)...

One way to interpret the URLs would be to walk them from the left
asking for each part that was a node name if it has some specific
method (say "node_handler") and if so, call "node_handler" passing the
remaining parts of the URL.

>>From there, a node such as "category" could use its "parameter" (in
this example "/technology") in whatever way it wanted.

Note that this is very close to controllers in the "standard MVC" web
implementations, which could be a good or bad thing... :o)

Andr=E9

_______________________________________________
Sputnik-list mailing list
Sputnik-list@lists.luaforge.net
http://lists.luaforge.net/cgi-bin/mailman/listinfo/sputnik-list

On Mon, Jan 19, 2009 at 3:56 PM, Andre Carregal
<carregal@fabricadigital.com.br> wrote:
> On Mon, Jan 19, 2009 at 1:19 PM, Jim Whitehead II <jnwhiteh@gmail.com> wr=
ote:
>> ...
>> I'm interested in the concept of being able to pass parameters to a
>> given node rather than always having to access a sub-node.  That could
>> open up a lot of really nice applications that right now aren't
>> terribly easy to accomplish.
>
> I also would like having such kind of "path processing". OTOH, this
> may impose a certain load on the dispatcher (I'm not sure how it is
> working currently)...
>
> One way to interpret the URLs would be to walk them from the left
> asking for each part that was a node name if it has some specific
> method (say "node_handler") and if so, call "node_handler" passing the
> remaining parts of the URL.

Currently the URL is processed from right-to-left, checking to see if
the full node name exists and if not checking to see if the "parent"
exists.  It continues this until it reaches a node that contains the
information necessary,

Perhaps the logic could be altered to check the full node node, and
process from left to right otherwise.  I'm not sure which is the more
likely use case, but I do agree this would give us quite a bit of
flexibility.

> From there, a node such as "category" could use its "parameter" (in
> this example "/technology") in whatever way it wanted.
>
> Note that this is very close to controllers in the "standard MVC" web
> implementations, which could be a good or bad thing... :o)
>
> Andr=E9
>

_______________________________________________
Sputnik-list mailing list
Sputnik-list@lists.luaforge.net
http://lists.luaforge.net/cgi-bin/mailman/listinfo/sputnik-list

On Mon, Jan 19, 2009 at 2:03 PM, Jim Whitehead II <jnwhiteh@gmail.com> wrot=
e:
>...
> Perhaps the logic could be altered to check the full node node, and
> process from left to right otherwise.  I'm not sure which is the more
> likely use case, but I do agree this would give us quite a bit of
> flexibility.

I guess a standard wiki would benefit from a full URL lookup (as the
current dispatcher does), but a more MVC oriented application would be
able to fully explore a left to right parsing.

I agree that we could check the full path just as today, and then try
the left to right approach, but I'd like to check what could happen
with the current collections processing. Yuri, have you already
landed? :o)

Andr=E9

_______________________________________________
Sputnik-list mailing list
Sputnik-list@lists.luaforge.net
http://lists.luaforge.net/cgi-bin/mailman/listinfo/sputnik-list

> I'm not sure if child defaults can handle patterns (I'd have to look)
> but if so you'd just set it so that any child node of /Tags is set to
> use a custom action for viewing that handles scanning the valid nodes
> in your wiki and displaying those with the correct tags (or
> categories).

They do, but you don't need it in this case.  It is easier to just use
child_defaults.any to capture all children, and then sort them out in
the action function.

Let me give a somewhat longer explanation. When you ask for foo/bar,
Sputnik first checks if there is actually a node called foo/bar. If
there is, then this node is sent the command. If there isn't, then we
look up node "foo" and look at a field called "child_defaults". This
field tells us what to do when a child of the node is requested but
does not exist. This field evaluates to a Lua table and has several
subfields. One of them is "any". If child_defaults.any is set, then
its value is returned for _any_ child (unless the child exists or a
more specific rule is matched first).

The value of child_default.any is a table that simulates a node.  For
example, suppose we create a node called "tags" and set child_defaults
to

    any = [[
      prototype="@tag"
    ]]

Then, a request for "tags/technology" will return a node that would be
blank, apart from the fact that it would have its prototype set to
"@tag" and also - and this is important - would have its "id" field
set to "tags/technology". In other words, the returned node will look
like this:

    node = {
      prototype="@tag",
      id = "tags/technology"
    }

The rest of the magic can go into the "@tag" node. For instance, we
can set the actions fields of @tag so that "show" is mapped to a
function show_nodes_matching_tag().  "tags/technology" will inherit
this action field from "@tag". All that is then left to do is to have
show_nodes_matching_tag() check the id field of the node that it gets
passed, and work based on that:

    actions.show_nodes_matching_tag = function(node, request, sputnik)
       local nodes = some_way_of_doing_a_query(node.id:match("[^/]$"))
       ...
    end

Now, onto actual filtering. From inside the action function, you have
access to the sputnik instance (passed in as the third parameter) and
thus to the saci repository. Sputnik wraps some of saci methods, so
let's assume you'll be using the wrappers when available. This
currently gives you two fully legit methods for accessing other nodes:

    another_node = sputnik:get_node(id, version)
    ids = sputnik:get_node_names{prefix="foo", limit=200}

This gives you a terribly inefficient way of looking for nodes by tag:
iterating through all of them. This works for a small number of nodes,
but it doesn't scale.

There is also a third method, which, however, is unofficial and will
likely be removed:

   nodes = sputnik.saci:query_nodes({"tags"}, "technology", "some_prefix")

This method is more like what you want, probably. The problem with it
is that it's wildly inefficient. It is basically almost as bad as
iterating through the nodes yourself, it just makes a few shortcuts.
Again, it works OK for a small number of nodes, and if you use a long
running process and cache the results, it can work OK even for a few
thousand nodes.  However, it's ugly.

To make such query work well, we would essentially need to reimplement
an efficient index. I am not sure I want to do this. I think a better
approach would be to use an existing index, perhaps come up with a
unified API for querying (within limits), and then integrate this very
thinly with Sputnik or Saci:

  hits = sputnik.indices.default:query("tags:technology AND ...")

Again, I'll be up for doing this integration, once we have bindings to
Xapian or something like that.

> I'm interested in the concept of being able to pass parameters to a
> given node rather than always having to access a sub-node.  That could
> open up a lot of really nice applications that right now aren't
> terribly easy to accomplish.

This _is_ possible. Selecting nodes based on the value of fields is
hard. But passing the id of one node to another is trivial.  For
instance, if we wanted to do something like translator/foo which
returns a translation of node foo into another language, we can do
this trivially using the method I described above.

- yuri

-- 
http://spu.tnik.org/

_______________________________________________
Sputnik-list mailing list
Sputnik-list@lists.luaforge.net
http://lists.luaforge.net/cgi-bin/mailman/listinfo/sputnik-list

> I guess a standard wiki would benefit from a full URL lookup (as the
> current dispatcher does), but a more MVC oriented application would be
> able to fully explore a left to right parsing.

We can have both, I suppose. The order for looking up foo/bar could be:

1. Check child_handlers field in foo. If it's defined, use it.
2. Check if foo/bar exists in storage. If so, use it.
3. Check child_defaults field in foo. If it's defined, use it.
4. If still nothing, return a blank node.

This _would_ require retrieval of an additional node, but apart from
that it would not be too demanding, since in a typical case we'll just
get "foo", check it's child_handlers, find nothing and come back to
today's strategy.

> Yuri, have you already landed? :o)

Yes. (I am back in California after a month in Brazil.)

- yuri

-- 
http://spu.tnik.org/

_______________________________________________
Sputnik-list mailing list
Sputnik-list@lists.luaforge.net
http://lists.luaforge.net/cgi-bin/mailman/listinfo/sputnik-list

On Tue, Jan 20, 2009 at 2:23 AM, Yuri Takhteyev <yuri@sims.berkeley.edu> wr=
ote:
> We can have both, I suppose. The order for looking up foo/bar could be:
>
> 1. Check child_handlers field in foo. If it's defined, use it.
> 2. Check if foo/bar exists in storage. If so, use it.
> 3. Check child_defaults field in foo. If it's defined, use it.
> 4. If still nothing, return a blank node.
>
> This _would_ require retrieval of an additional node, but apart from
> that it would not be too demanding, since in a typical case we'll just
> get "foo", check it's child_handlers, find nothing and come back to
> today's strategy.

+1

Andr=E9

_______________________________________________
Sputnik-list mailing list
Sputnik-list@lists.luaforge.net
http://lists.luaforge.net/cgi-bin/mailman/listinfo/sputnik-list

On Tue, Jan 20, 2009 at 11:43 PM, Andre Carregal
<carregal@fabricadigital.com.br> wrote:
> On Tue, Jan 20, 2009 at 2:23 AM, Yuri Takhteyev <yuri@sims.berkeley.edu> wrote:
>> We can have both, I suppose. The order for looking up foo/bar could be:
>>
>> 1. Check child_handlers field in foo. If it's defined, use it.
>> 2. Check if foo/bar exists in storage. If so, use it.
>> 3. Check child_defaults field in foo. If it's defined, use it.
>> 4. If still nothing, return a blank node.
>>
>> This _would_ require retrieval of an additional node, but apart from
>> that it would not be too demanding, since in a typical case we'll just
>> get "foo", check it's child_handlers, find nothing and come back to
>> today's strategy.
>
> +1

I would argue pretty heavily against doing anything before checking to
see if the full node exists.  I don't see what the above gains us that
we don't have by swapping 1 and 2.  I agree with the overall change in
strategy.

- Jim

_______________________________________________
Sputnik-list mailing list
Sputnik-list@lists.luaforge.net
http://lists.luaforge.net/cgi-bin/mailman/listinfo/sputnik-list

> I would argue pretty heavily against doing anything before checking to
> see if the full node exists.  I don't see what the above gains us that
> we don't have by swapping 1 and 2.  I agree with the overall change in
> strategy.

I am not sure what you mean. If we switch 1 and 2 then we are
essentially back to where we are right now, unless there is a clear
difference between child_handlers and child_defaults. (If they both
happen after checking the node, then they might as well be merged.) So
there is not change in strategy. Which perhaps is fine.

I think we need specific use cases before we understand if a change is
worthwhile at this point. If we cannot come up with specific examples
of things that we cannot do with the current design, then I would
stick with what we have for now.

 - yuri

-- 
http://spu.tnik.org/

_______________________________________________
Sputnik-list mailing list
Sputnik-list@lists.luaforge.net
http://lists.luaforge.net/cgi-bin/mailman/listinfo/sputnik-list

Let me look a little more closely at the current implementation and
think about before continuing this thread. This week is quite busy,
but hopefully over the weekend or next week...

  - yuri

> To be a bit clearer, hopefully, I'm not sure I understand why we're
> considering such a change in behavior when we can add parameters like
> we're discussing without altering the way things currently work.

-- 
http://spu.tnik.org/

_______________________________________________
Sputnik-list mailing list
Sputnik-list@lists.luaforge.net
http://lists.luaforge.net/cgi-bin/mailman/listinfo/sputnik-list

> I prefer that the entire node id is always checked first. Its fast and
> if that node exists I would hope that you'd want to serve it. Failing
> that we can check the root node and work our way in the chain from
> left to right. Certainly even this changes the semantics of node
> processing now but I do not believe we have any cases where this would
> break anything.

So, the only thing I don't understand is why left-to-right?
Right-to-left means "if the node doesn't exist, ask its immediate
parent". This gives us natural recursion.

> My primary disagreement is not checking to see if the full node exists
> before taking any other action.  It's the cheapest thing that we can
> do and it's the default case in a non-application heavy wiki.

Yes, I agree on this. I thought that this is we had already. But I was wrong.

I fixed this and it now does this:

    saci:get_node(id):
         parent, rest = string.match(id, "^(.+)/(.-)$")
         first try versium:get_node(id)
         otherwise try saci:get_node(parent):get_child(rest)

This means you start from the actual node, then fall back onto its
immediate parent, then at the parent's parent, etc. Nice and
recursive, no?

- yuri

-- 
http://spu.tnik.org/

_______________________________________________
Sputnik-list mailing list
Sputnik-list@lists.luaforge.net
http://lists.luaforge.net/cgi-bin/mailman/listinfo/sputnik-list

On Mon, Jan 26, 2009 at 7:49 AM, Yuri Takhteyev <yuri@sims.berkeley.edu> wr=
ote:
> ...
> This means you start from the actual node, then fall back onto its
> immediate parent, then at the parent's parent, etc. Nice and
> recursive, no?

Nice and recursive. But still right to left... :o)

What about going all the way to the left and then trying to the right,
before giving up and reporting a "node not found"?

Andr=E9

_______________________________________________
Sputnik-list mailing list
Sputnik-list@lists.luaforge.net
http://lists.luaforge.net/cgi-bin/mailman/listinfo/sputnik-list

> Nice and recursive. But still right to left... :o)

Can you give an example of something that cannot be done with the
approach I described?

Note that with this approach, if you ask for 'foo/bar/baz/quux' and
'foo' has no children, then the node does get resolved in the order in
which you would expect it to: 'foo' is asked to provide 'bar', which
is then asked to return 'baz', which is then asked for 'quux'.

Note that if you start with just 'foo' you can configure it in such a
way as to ensure that it would never have any real children. To create
'foo/bar' the user would need to ask for 'foo/bar', then edit it, then
save it. Since 'foo/bar' doesn't exist, what the user will edit is the
default version of 'foo/bar' provided by 'foo'. This means 'foo' gets
to control whether new children are going to be created, and if so,
how they can be saved.

The reason I want to do the original check from right to left is to
make it easy to put nodes with default children at any level in the
tree. That is, if 'foo' is configured to handle 'foo/bar', then it
would be nice to be able to rename 'foo' to 'stuff/foo' and have
'stuff/foo/bar' still work.

- yuri

-- 
http://spu.tnik.org/

_______________________________________________
Sputnik-list mailing list
Sputnik-list@lists.luaforge.net
http://lists.luaforge.net/cgi-bin/mailman/listinfo/sputnik-list

On Tue, Jan 27, 2009 at 6:49 AM, Yuri Takhteyev <yuri@sims.berkeley.edu> wr=
ote:
> Can you give an example of something that cannot be done with the
> approach I described?

With your recent explanation, no. :o)

> Note that with this approach, if you ask for 'foo/bar/baz/quux' and
> 'foo' has no children, then the node does get resolved in the order in
> which you would expect it to: 'foo' is asked to provide 'bar', which
> is then asked to return 'baz', which is then asked for 'quux'.

Right, seems to work pretty fine.

> Note that if you start with just 'foo' you can configure it in such a
> way as to ensure that it would never have any real children. To create
> 'foo/bar' the user would need to ask for 'foo/bar', then edit it, then
> save it. Since 'foo/bar' doesn't exist, what the user will edit is the
> default version of 'foo/bar' provided by 'foo'. This means 'foo' gets
> to control whether new children are going to be created, and if so,
> how they can be saved.

Nice!

> The reason I want to do the original check from right to left is to
> make it easy to put nodes with default children at any level in the
> tree. That is, if 'foo' is configured to handle 'foo/bar', then it
> would be nice to be able to rename 'foo' to 'stuff/foo' and have
> 'stuff/foo/bar' still work.

I understand the importance of being able to do this kind of move, but
I don't see why it would be restrained from working since in this case
"stuff" would have to exist as a real node, right?

Andr=E9

_______________________________________________
Sputnik-list mailing list
Sputnik-list@lists.luaforge.net
http://lists.luaforge.net/cgi-bin/mailman/listinfo/sputnik-list

> I understand the importance of being able to do this kind of move, but
> I don't see why it would be restrained from working since in this case
> "stuff" would have to exist as a real node, right?

It doesn't. Only 'stuff/foo' must exist. The order of processing will
be the following:

Does 'stuff/foo/bar' exist? No.
Does 'stuff/foo' exist? Yes. Ok, then get 'stuff/foo', ask it for
'stuff/foo/bar'.

So, we never get to asking for 'stuff'. If we wanted to let 'stuff'
handle 'stuff/foo', then we would need to remove 'stuff/foo'. Then we
would have:

Does 'stuff/foo/bar' exist? No.
Does 'stuff/foo' exist? No.
Does 'stuff' exist? Yes.
Ok, then get 'stuff', ask it for 'foo', ask stuff's 'foo' for 'bar'.

Practically, btw, you would want to use prototypes to achieve this
kind of chaining. 'stuff' could define the default for it's children
as {prototype="@Child_of_Stuff"}.  So, a request for "stuff/foo" will
then return {id="stuff/foo", prototype="@Child_of_Stuff"}. The
handling of "stuff/foo/bar" would then be essentially determined by
the child_defaults value in @Child_of_Stuff.

- yuri

-- 
http://spu.tnik.org/

_______________________________________________
Sputnik-list mailing list
Sputnik-list@lists.luaforge.net
http://lists.luaforge.net/cgi-bin/mailman/listinfo/sputnik-list

On Wed, Jan 28, 2009 at 6:40 AM, Yuri Takhteyev <yuri@sims.berkeley.edu> wr=
ote:
>> I understand the importance of being able to do this kind of move, but
>> I don't see why it would be restrained from working since in this case
>> "stuff" would have to exist as a real node, right?
>
> It doesn't. Only 'stuff/foo' must exist.
> ...

I see, thanks again! :o)

Andr=E9

_______________________________________________
Sputnik-list mailing list
Sputnik-list@lists.luaforge.net
http://lists.luaforge.net/cgi-bin/mailman/listinfo/sputnik-list