ActivityStreamPlugin Development

This is the topic to discuss development of ActivityStreamPlugin

If you need support, go to Support.ActivityStreamPlugin where you can ask questions and find answers to previously asked questions. If you want to report a bug, or a feature request, go to Tasks.ActivityStreamPlugin where you can see already submitted issues and where you can submit a new bug report or feature request.

Active Items

Id	Summary	Priority	Current State	Creation Date	Last Edit
Item11964	initial work on an activity stream and microblogging system for foswiki	Enhancement	Being Worked On	21 Jun 2012 - 15:13	12 Dec 2017 - 17:00

Examples

Links

• http://activitystrea.ms/: Activity Streams - a format for syndicating social activities around the web
• http://wiki.activitystrea.ms: ActivityStreams Community Wiki

Foswiki events and how they are processed in the ActivityLogger

Raw event	Message	Mapped action	Rewritten Message
Core
*	scumbag/AUTOINC/XXXXXXXXX/	ignored
attach		ignored
changes		ignored
edit		ignored
edit	(not exist)	new
login	SUCCESS - user	login
	FAIL - user	loginfailure
logout	LOGOUT - user	logout
move	web.topic.attachment move(d) to web.topic.newattachment	renameattachment	target=newattachment@web.topic
move	web.topic.attachment move(d) to otherweb.othertopic.attachment	moveattachment	target=attachment@otherweb.othertopic
move	source move(d) to trash web	deleteattachment
rdiff	int int	compare	rev1=int rev2=int
register	user@domain	register	user
rename	move(d) to web.topic	rename	target=web.topic
rename	move(d) to trashweb.topic	delete
renameweb	move(d) to web	renameweb	target=web
renameweb	move(d) to trashweb	deleteweb
rest		ignored
save		edit
save	attachment	editattachment
search	pattern	search	pattern
upload	filename	upload
view
view	(not exist)	ignored
viewfile		ignored
MetaCommentPlugin
comment	state=... title=... text=...	comment	state=... title=... text=...
commentapprove	state=... title=... text=...	commentapprove	state=... title=... text=...
commentdelete	state=... title=... text=...	commentdelete	state=... title=... text=...
commentupdate	state=... title=... text=...	commentupdate	state=... title=... text=...
CopyContrib
copy
CompareRevisionsAddOn
compare		compare
SqlPlugin
sql	sql statement

Grouping messages in the ACTIVITYSTREAM widget

activities are extracted from the logs data base grouping them by

• user
• action
• timerange (1 hour by default)

Actions of the same user ocurring in the given timerange are pooled and formatted indepentently. Some actions don't require a "..._multi" variation. Some actiosn are ignored by the ACTIVITYSTREAM widget. These kind of informations are still kept in the logs database and may be reused otherwise. For instance view actions are not displayed as part of the activity stream, although a view statistics is interesting to display such as:

• per page: how often has this page been visited (within a certain timerange)
• hot topics: which topics are visited most often

However this is out of the scope of the ACTIVITSTREAM widget itself and requires a different widget to be implemented.

Ignored actions:

• compare
• loginfailure
• register
• view

Friday, 11. February 2011

rename

rename_multi

delete

delete_multi

renameweb

renameweb_multi

deleteweb

deleteweb_multi

editattachment

editattachment_multi

search

search_multi

upload

upload_multi

comment

comment_multi

commentapprove

commentapprove_multi

commentdelete

commentdelete_multi

commentupdate

commentupdate_multi

-- MichaelDaum - 21 Jun 2012

Discussion

I'm concerned that Foswiki logging is not sufficient. I've been finding places where modifications are completely missed in the logging. For ex. the Groups API did no logging at all in the events log. I suspect core really needs an in-depth audit to make sure that things don't slip through the cracks:

• Low level Store event logging
  • Record every modification to store, topics and attachments
  • Optional - access logging of ever read of a store object
• Event logging
  • Audit all APIs and scripts, ensuring that every possible path for logical modificatiions of topics is recorded.

-- GeorgeClark - 21 Jun 2012

Nifty!

-- PaulHarvey - 22 Jun 2012

Agree with George about the audit. The starting point is a comprehensive list of "interesting events" - which Michael has created above, and I think we should adopt as a gold standard.

-- CrawfordCurrie - 22 Jun 2012

Oliver and I met in Hamburg 2 days ago and discussed this in great detail.

Of course event logging isnt enuf by far. These are only a quick way to get any content into a database. The events that the core emits will be added to the activities database as only one "collector" of many. The plugin will offer abstract classes and interfaces to register collectors for different kind of events and actions.

Likewise the way things are verbalized right now wont stay as it is. This too has to be made pluggable.

So we will have:

• (a) collectors: base classes and an api to implement custom event types to be added to the database
• (b) verbalizer: an api to produce a localized human readable representation of a single event or a grouped bunch of events that end up as one new row in the activity stream.
• (c) search & filter: assess raw data in database, apply filtering and grouping and call the appropriate verbalizer for the type of event to be rendered
• (d) renderer: produce the desired output format: html, json, rss, atom

The database will log events and actions in a kind of tripple as well as a set of meta data describing properties of object and subject (work group, team, category, tag, acl):

1. subject: the person that produced that event
2. verb: the action that the subject performed on the object
3. object: the affected object (e.g. a topic, attachment, comment, tag, category, formfield, a custom foo-bar thing)

The verbalizer will have to be able to cope with different kinds of data being grouped together to be uttered in one. These are:

• singular event: something happened only once
• multi event: a couple of events happened in a short period of time done by the same subject, or the same object. (e.g. "You and five others commented on topic x.", "John doe approved the invoice FooBar and five others ... check out details.")
• second person: "You did this"
• third person: "John Doe did this."
• stress a specific object in a multi-event: "John Doe uploaded this attachemnt and five others ... click here for details".

There are two kinds of activity collectors that one might think of implementing:

• automatic events: these are detected automatically as a consequence of the user interacting with the system independent of any direct interaction with an activity stream
• manually produced events: like prominently microblogging -> a user manually records an event that happened somewhere else outside of the system

Putting up a poll or asking a question is a manual event, as well as responding to it. These require a verbalizer of their own as well ("John doe asked a question/ answered a question/put up a survey/added his 2cent to the poll).

The key is that the base is pluggable to add things like polls, commenting, question-answering as an add on that can hook into the activity stream to be recorded and rendered appropriately.

A side issue is user profiles. Foswiki doesn't formalize what the avatar/photo of a user is, nor what the thumbnail representation of any other topic type is. We need this visualizing search results not only in activity streams. User photos are very important with regards to the way people perceive and interact with an activity stream. You will virtually see who did something on the wiki. That's quite a different feeling towards the system and the stuff happening. It is much more oriented towards the human beings behind the data being stored rather than the data that by the way was changed by someone. So it is in some sense the inverse of WebChanges, properly grouped and "humanized" the way the data is presented.

This kind of architecture isn't part of the current implementation by far. It is more a kind of getting the feet wet thing to get a feeling about the domain.

The plugin does work and is fully functional in that you can see data being displayed in System.SiteActivityStream and WebActivityStream per web. There's some more filtering already there to display only a certain group's activities or types of activities. Looks like the mocked up example above. Existing core events can be imported using the import rest handler. There's a Foswiki::Logger::ActivityLogger impementation that feeds data to the database while still having the plain file logger as a secondary one to bootstrap the database from scratch any time.

With regards to the current way to "reuse" the Foswiki::Logger gathering what is there, there's a lot of clean up required to make any sense of the data coming in that way. The most valuable source of information which kind of event actually stands behind a certain log message is ... as usual ... stored in the comment field. Some of this information might probably be too unspecific to really deduce the kind of event that actually happened. I did not make it the full way down all of those log messages as you can see above.

So it would very well help to rationalize the format of the log messages that we have right now, and even add some more here and there. Some are quite pointless and completely useless from the POV of an activity stream (see the ignores above). Well, but we can live with the current kind of log message too that the core emits right now. If there isn't an event being recorded then it wont be part of the activity stream. Most of the important stuff is already in there like create, delete, edit, move things around. Point is that we can isolate these problems in their own activity collector classes feeding data to the database. When the log format of the core events changes, only one class has to be fixed but not the complete plugin architecture.

-- MichaelDaum - 22 Jun 2012

http://activitystrea.ms/ and http://activitystrea.ms/specs/json/1.0/ may be a worth reading.

It describes an exchange format for activity streams in json and/or xml.

-- OliverKrueger - 22 Jun 2012

Maybe of interest aswell: http://pump.io/

-- OliverKrueger - 15 Jan 2013

Commenting is disabled while not logged in