I did a release of Demisauce (Source code on github) recently and wanted to show off what it is, so include the capabilities for Comments, Polls, and the Help/Idea/Feedback badge on my Potlatch Blog code (hosts this site). Also, the demo Django app shows how to use these services.
With Demisauce.com now hosted on Amazon EC2 I am more confident it can handle the load and I will look to start using it for higher volume applications, so if you are interested in using it please go to the signup page and enter your email. I am only allowing a few signups for right now to ensure the servers have capacity but will allow a few to start.
Comments I know that Disqus.com is a great comment service, currently vastly superior to the one offered here. BUT, the difference I believe is use case. Demisauce comments are meant to be embeded within an application more like native functionality or added to corporate apps whereas Disquis is more of a consumer oriented (IMHO) service. By embedded I mean used as a component for development, the component happens to be a service, but allows for distributed application development.
Feedback Badge Again, there are great services like Uservoice.com, and Getsatisfaction.com that offer great feedback. However, the use of Javascript as integration is convenient and easy but can cause performance issues, and you would still need to do user integration. These seem to be catering to consumer oriented applications. The applications I want to use them on at work would be corporate employee applications. I see the API's and ability to have private have improved at UserVoice and Getsatisfaction, however maybe it just comes down to wanting open-source products as an option. Also, the ability to use one identity integration.
Polls This is a simple polls service, however I believe the big advantages are: Open-source, more able to be embedded in corporate applications, but also it is a able to be cached (memcached, or your local cache) so that client side javascript is not a performance issue.
How to EmbedHere is an example of how to embed the poll (in Django app, there are php examples in the demos folder).
{% dsview "feedback" "badge" %}
From source.. Or
Service definition on demisauce.
I have the Activity Graphs also ready to go, but have not created the Services API for how to use them so will get those next time along with an article on how to embed these services in your app.
Its been a long time, but I have been working hard on getting another version of demisauce out. This time the focus was on moving the hosting to Amazon EC2 and getting my Blog updated to utilize it.
So, whats new?
- Installer, able to install easily on vmware image, or on EC2
- Service Registry, a very webby simple service registry. Able to have other users add plugins (services), and utilize them without new libraries or installs.
- New CSS theme on the blog
- Support for Google App Engine, changes to the library to support GAE memcached and fetch url.
- Django Demo a simple demo to show Demisauce consumed and used within a Django App.
I was working on a way to host my Sphinx generated documentation, and realized that Google App Engine was perfect for it! Not only is the hosting Free, but it has a great automatic deployment tool that will upload all the documentation. So, the first Demisauce Documentation is now online. You can check out the Code necessary to support this. It is just a few lines of app.yaml configuration and a very simple redirect.
A friend was asking "what is Demisauce", of which i have been horrible at articulating, so here goes an attempt:
Demisauce is a set of Services, and a Services framework
- A set of services to use within your app (as service oriented/easy to integrate javascript or xml): "feedback/help/ideas", activity tracking (per person google analytics), comment system, poll's, group management. All very boring EXCEPT: these are all services able to be plugged in to your app instead of having to build on top of wordpress/joomla/sharepoint to get access to them.
- A plugin services framework: think service oriented Wordpress plugin's, again available to any app for plugin. (cloud service plugin framework). The goal was to create an plugin based open source tool, that cater's to developer's adding in their own plugin's. However, since it was service oriented, a services framework was needed. Instead of having each developer integrate with 50 api's (see below) this provides a common place to exchange services similar to Opensocial gadgets, or Google gadget's but not based on javascript client side integration (slow, slow, slow).
- Integration as a service: As a side product of moving to a cross-application, cross-data-center services framework you can now provide integration as a service. Think of an "add user" web service that you call, you have previously configured through web based admin what you want to happen: "when new user gets added, add them to: (wordpress, google apps, salesforce.com, basecamp, etc,etc)". Think of having one api for add user, behind which is integrations' with services so instead of each developer/company doing integration with google apps, Salesforce.com, Wordpress, instead you call this one and it has pre done integration. The idea, as more and more cloud services pop-up you are going to need to register your user's on those systems. Why have each company do 50 "add user" integration services? Instead of integrate with one, then it integrates with the 50?
I have been quiet but still making lots of updates to Demisauce. Most recently i have updated the Help and Feedback processing system. It is somewhat of a light scale embeddable app that can collect feedback and suggestions, ideas, and trouble from your user's.
It shows off several of the new features i have been working on. The biggest of which is the tagging system. I am working on extracting the tagging as a fully embeddable service (json, javascript, xml, html api's).
So, this help system is definitely not a full blown ticketing system. It is geared towards several things: More towards feedback, and easy to embed. If you are looking for full blown open source trouble ticket or help systems, here are a few: http://www.eticketsupport.com/, Simpleticket.net, OTRS.org, TroubleTicketExpress.com, http://www.osticket.com/, http://www.hulihanapplications.com/projects_mystic.php.
- Easy To Embed in Your App: Using Javascript, or xml api's easy to put collection of user's input. Since it is service oriented, you can serve many app's with the same implementation (hosted on Demisauce.com soon) or install on your servers.
- Feedback Processing: It is geared more towards feedback and suggestions by end user's. So, you can put a link inside your footer or in your website to ask for feedback. It pop's up a form that is hosted on Demisauce, allowing zero install implementation's other than javascript. But, more XML api's are available if needed. Currently tagging works great, working on ability to assign item's to other user's as task's that send notifications.
- Event Driven Architecture: Currently being worked on is a set of Event callback api's. So, you can recieve call back via web service api's or simple Web Hooks API.
We have been working on building some Google Gadget's at work as a way to do distributed application development. We are looking to use Shindig, the Open source Gadget server. To interface with it we were looking into Thrift, a cross-language services framework. However, to get it working it has quite a few non-trivial dependencies, and not the best documentation. So, anyway, I followed This article on installing Thrift, however i received the following error:
---> Building boost with target all
Error: Target org.macports.build returned: shell command " cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_devel_boost/work/boost_1_35_0" && gmake all " returned error 2
Command output: Makefile:2: *** missing separator. Stop.
Error: Status 1 encountered during processing.
Some more googling led me to
This MacPorts trac ticket which has a mention from someone on getting it working (with the Clean, then reinstall steps).
That
Nearly got me there, but i ran into this error.
checking for libevent >= 1.0... yes
checking for zlib >= 1.2.3... yes
./configure: line 20726: syntax error near unexpected token `MONO,'
./configure: line 20726: ` PKG_CHECK_MODULES(MONO, mono >= 1.2.6, have_mono=yes, have_mono=no)'
mac:thrift aaron$ sudo make install
Password:
make: *** No rule to make target `install'. Stop.
mac:thrift aaron$ sudo port install mono
Some more googling led me to this
http://wiki.apache.org/thrift/ThriftInstallationWin32, the section about the missing .m4 file. My .m4 file was actually in /opt/local/share/acloal so i modifed it for the following:
cp /opt/local/share/aclocal/pkg.m4 aclocal
I was still getting the same missing mono, error, some more googling led me to
http://blog.jakbot.com/?p=6 I did couple things this page, but then re-ran
sh bootstrap.sh
I think that is what did it, although if not I also followed the steps in the above article about exporting mono path, and also the additional installs (autconf,automake, libtools). Then, i was able to re-run the Thrift .configure command and it was good.
While playing with Thrift, i also found ThruDB which sounds like a very interesting service as well. I started to follow this article on installing ThruDB. I am tackling that now. Have to start to find the Python API's.
The newest version of Demisauce in svn includes a new activity tracker (I will blog in more detail in the future). This activity tracker utilizes the Google Visualization API on a new Dashboard (screenshot below) to present visual graphs of user activity. Big deal, its not as nice as Google Analytics you say? Well, true, but it does allow you to track information about users so that you know the exact usage for a particular user, instead of anonymous statistics for all users.
However, while playing with the Visualization API, which I found to be a great product and a great abstraction, I really was wanting some different graph formats. So, I went in search of some different visualization API's in Flash which were hopefully opensource, here are some i found. I am maybe going to dig in and make a treemap version of a Google Visualization API? We will see....
A lot of work on the Demisauce Help system, using facebox jquery plugin for the popup page, or at least a heavily modifed version of Facebox. I had to implement quite a few changes to allow for content to be gathered from the Demisauce server, which would be a different domain. Therefore, the traditional ajax get in the Facebox wouldn't work, and i had to modify it to do a Document.write instead.
I added this method to Facebox to route requests via an inserted script.
// in facebox.js
function fillFaceboxFromDocumentWrite(href, klass) {
if ($('#facebox .content').attr('src') == href){
$.facebox.reveal(null, klass,href);
}else {
var ds_html_script = document.createElement('script');
ds_html_script.type = 'text/javascript';
ds_html_script.src = $.facebox.settings.base_url + href;
document.getElementById('facebox').appendChild(ds_html_script)
$.facebox.reveal(null, klass,href);
}
}
The script that gets called is dynamically generated to produce javascript, which will get inserted into the Dom of the remote server.
// python code to generate ds_help_output
document.getElementById('facebox_content_holder').innerHTML = ds_help_output;
The end result is this help system. You can edit content at the Demisauce Hosted Server or can host a version of demisauce, then include a line of javascript in your apps to get built in help system. Here is a screenshot.
Also, modified Demisauce to allow for Comment notification's.
I made a number of improvements to this blog engine over the last couple of weeks. Most note-able was performance improvements to pre-render and cache HTML. There has been a lot of discussion and movement in the web architecture community towards not-traditional Database designs, not normalizing data, and storing caclulated info, and even moving to non-relational database designs. All because Internet scale data-bases built on transactional systems just seem fraught with problems and forces you to build around the db instead of around the needs of the web apps. The Google App Engine article on High Scalability gives a lot of great insight into App Engine which i had been discovering separately.
In order to incorporate a lot of the improvements possible out of Google App Engine Datastore, you have to start to do things different, here are what i did to help build a better google app engine based app:
- OnChange Handler: Since we are often pre-rendering, and caching html, we need to get hooked into the events which signify when we need to make a change. Digging into the Google Datastore API, i did some reverse engineering to try to figure out how I can figure out when something is changing. What i came up with is to provide a Base model which i inherit from which allows me to write OnChange methods per attribute.
from google.appengine.ext.db import Model as DBModel
class BaseModel(db.Model):
def __init__(self, parent=None, key_name=None, _app=None, **kwds):
self.__isdirty = False
DBModel.__init__(self, parent=None, key_name=None, _app=None, **kwds)
def __setattr__(self,attrname,value):
"""
DataStore api stores all prop values say "email" is stored in "_email" so
we intercept the set attribute, see if it has changed, then check for an
onchanged method for that property to call
"""
if (attrname.find('_') != 0):
if hasattr(self,'_' + attrname):
curval = getattr(self,'_' + attrname)
if curval != value:
self.__isdirty = True
if hasattr(self,attrname + '_onchange'):
getattr(self,attrname + '_onchange')(curval,value)
DBModel.__setattr__(self,attrname,value)
Then, on my Entry class (blog entry), i implemented a method whenever an entry changes from published to draft and back, and updated the archive.
class Entry(BaseModel):
author = db.UserProperty()
blog = db.ReferenceProperty(Blog)
published = db.BooleanProperty(default=False)
content = db.TextProperty(default='')
# more cut out
def published_onchange(self,curval,newval):
"""
Gets called every time published status changes
"""
if self.entrytype == 'post':
my = self.date.strftime('%b-%Y') # May-2008
archive = Archive.all().filter('monthyear',my).fetch(10)
if curval == False and newval == True:
# add to archive
if archive == []: # new month
archive = Archive(blog=self.blog,monthyear=my)
else:
archive = archive[0]
archive.entrycount += 1
archive.put()
self.blog.entrycount += 1
else:
# remove from archive
if archive and archive[0]:
archive = archive[0]
archive.entrycount -= 1
if archive.entrycount == 0:
archive.delete()
else:
archive.put()
self.blog.entrycount -= 1
self.blog.save()
- Pre-Render HTML: On the previous entry, we are storing some tables (Archive) which are purely used for convenience. They are never called at runtime, and in fact the Archive table is then rendered into html and stored again. Here is an admin entry "POST" update that then calls to update the cache. The Cache is actually a Blog entity which is used on every single request. It is half way between a Cache, configuration and context in normal web apps.
class AdminEntry(BaseController):
def post(self,entrytype='post',key=None):
#update entry from form post
entry.save()
rebuild_cache(self.blog)
def rebuild_cache(blog):
"""
Pre-Render's and cache's html in blog object. Everything
in here doesn't change very often, so we can update it at point of change
"""
pages = Entry.all().filter("entrytype =", "page").filter("published =", True).fetch(20)
archives = Archive.all().order('-date').fetch(10)
recententries = Entry.all().filter('entrytype =','post').filter("published", True).order('-date').fetch(10)
links = Link.all().filter('linktype =','blogroll')
template_vals = {'recententries':recententries,'pages':pages,
'links':links,'archives':archives}
path = os.path.join(os.path.dirname(__file__), 'views/sidebar.html')
blog.sidebar = template.render(path, template_vals)
path = os.path.join(os.path.dirname(__file__), 'views/topmenu.html')
blog.topmenu = template.render(path, template_vals)
blog.save()
These tricks helped make writing App engine even easier, moving more to an event driven model that updates the cache. You can find all the code from this app at the Github code repository.
Another release of
Demisauce including bug fixes, and the start of a help system.
Here is a screenshot of the Help System which is an popup over the rest of the page. There are a couple of different styles. The content comes from Demisauce, and both the help (javascript) and its content come from Demisauce server and are easily embeddable in an app.
