BlueIgnis In Making - Pre Alpha Preview

I wonder how many have you experienced what I had for the past 2 days. I had a phone call from my project co-ordinator stating that I might not get my degree. There has been a lot of mis-communication by the person involved in the issue. Shit Happens!! Now I have my 2nd and 3rd Review on Monday, I consider myself special, you know ;-)

I never really started to build something for my project until Thursday when I had the phone calls. I shifted all my engagements at work till Tuesday to get into the damn thing.

Scenario - I had 2 days for building a demo, 1 day for preparing the documentation, and 1 day for travel (which I generally do at night to save time).

To my own astonishment I was able to come with a demo-able something in 1 day, which I shared with some colleagues of mine to improvise on demand and fix some bugs. After around 1.5 days of work, here are some of the screen shots from the application.

Ingredients^# - PHP (as always) + MySQL + Redis + Twitter Bootstrap + jQuery +
Coke + Pressure to get a Degree (*wink*)

# Prioritized on usage

First the Dashboard, it shows you the overall stats of the Twitter feeds and their sentiment division among Positive, Negative and Neutral. Currently I have implemented only Twitter feeds, Facebook and G+ should not take much time to implement.

Next is the Key Influencers. People who shape the conversation about your product / event / campaign. It is shown in a network form, where the center node is a picture that represents your campaign, and all other nodes represent the users. User's contribution towards the campaign denotes the higher size of the node. One of my friends suggested to improvise on this using Klout score too, seems like a very nice idea, need to see if it is possible and it would be a Kicking Ass feature if implemented.

Next comes the Realtime Twitter Dashboard. As we get in Tweets to the system, we push them to the screen so that users see what people are talking about their campaign in real time. I do not know UX or anything related to UI, but I am a freaking good user.

I know the problems user's might face and one such problem was. When I kept on adding tweets to the page in realtime, for a high frequency campaign. After around 10 - 15 min, my browser started becoming slow / non-responsive. Too much of content on page was not at all a good idea, hence now we start off with 10 tweets pre-loaded into the page and it gets updated in real time. Still there are no more than 100 tweets at the page (on either side) at any given point of time. Having the browser and page more responsive.

I am still open for any further improvements / changes by an expert. 

Next comes the Realtime Sentiment Dashboard  - Real time feed of sentiments for the campaign. As we process the tweets as Positive / Negative they drop by into the screen in real time.

Apart from this we have a login screen and a create campaign page which helps you create campaigns. Once I am done with this documentation I really wish to spend some considerable amount of time on this to add more features and analytics to it.

Would you be interested on using it? Do you think there are additional features that you might be interested in seeing in this? UX / UI tips for the novice here? Any suggestions are always welcome and drop by a comment stating what you feel about it. 

Twitter Streaming Limit Workaround

I was working on my final year project (BlueIgnis) which uses Streaming Twitter API. I had the following understandings from them (on free version):

1. One Account can open only One Streaming Connection at any given time
2. One IP may be associated with only One Account while streaming. Rotation of Streaming connections based on multiple accounts are not allowed. May lead to IP Ban. (All the more reason to use EC2 Instances for Streaming :P)
3. One Streaming connection may allow upto 400 tracks (different keywords) to filter from.
4. Reply to 402 Error codes with proper HTTP Status.
5. Should use non-aggressive re-connect policies, must give substantial amount of time in-between subsequent requests.
6. Periodically we must stop the Streaming Connection, add more tracks (keywords) to the list and re-start the connection, rather than individual connections for multiple times.

Based on these understandings, I came up with own Architecture for Twitter Streaming. Below diagram represents the overall architecture of my application with respect to Twitter Streaming Component.

Hosting the Twitter Streaming on an EC2 Instance, we can achieve 400 tracks (keywords) per node which can handle approx. 30 - 50 customers based on my use-case. I periodically (~10 min) check if there are any new tracks that needs to be added to the node until it becomes 400. Since I need to know which user requested the track, which is not possible to get from the current way the Streaming API works. 

So I decided to build a Local Firehouse, where in I stack all the tweets for all the tracks, all in a single location. Then, I use a FullText Search feature of MySQL (my datastore) to search for the related tweets continously so that I can achieve the feel of a bit delayed streaming yet close-to-realtime processing.

If you have any better ways to get things done, please let me know.

BlueIgnis - Starting Finally

Finally after almost 8 months of planning, modelling, and designing process - BlueIgnis is finally taking place. Good part is that my mentor has asked me to do Oracle ThinkQuest as the final year project. I was hunting for a good topic to build. The category I am eligible is "Application Development", and when I was wondering I decided its time I spend some time for BlueIgnis (aka Social Heat).

Taking the wonderful experience from building a custom framework using simple design patterns (more on this in a later post) for Webnaplo, and from the previous TGMC project (Back To My Village) - Reformists; Past 2 days went good building the BlueIgnis architecture with more new features which I only used to dream before.

I am all excited to work on this. This time with more features packed into its design. Since I am planning to release this as a standalone app and not as SAAS (I wish I can do it), it runs on your infrastructure.

For people who does not know what BlueIgnis is, you can refer my earlier blog post which gives a gentle introduction about its functionality.

Keep watching this space for more information and updates regarding BlueIgnis.

Social Heat - My Social Analytics System

Here is the gentle introduction of Social Heat - Social Analytics Engine.

If you have used Google Analytics service, you might get an idea what exactly I'm trying to say here. Unlike Google Analytics, which gets you detailed analytics for your website visits. Social Heat, gives you the detailed analytics report of your product on Social Networks (like Facebook. Google Buzz, Twitter, etc.)

Inspired by the following video for Viral Heat, I decided to spend this summer vacation creating a system on par or more sophisticated than Viral Heat.

So, I decided why not create one for this summer. Currently Viral Heat service is not free, but its really cheap starting at only $9.99 per month. It comes with unlimited impressions for your profiles.

Social Heat, following the philosophy of Google, will allow free usage which should suffice more than enough people around their world. It also has paid plans which will be disclosed later. I plan to provide more services than Viral Heat, which will be rolled out eventually following the public release.

Features of Social Heat:

1. Featured report on Users world wide from Facebook, Twitter, Google Buzz, YouTube and Blogs (blogging is also a part of the social media)
2. Export your reports in one of the many formats (CSV, Excel, PDF, XML, etc.)
3. Create profiles and add Mentions to it. Each Mention comes with unlimited impressions. This feature was inspired from Google Analytics and Viral Heat dashboard.
4. Social Heat has inbuilt machine learning capabilities. If you ever think an impression is not related to your needs, you can always teach the system not to display such kind of feeds again. Similarly, the reverse - If the system has not found an impression to be related to your product, you can add them to your report. System is completely dynamic.
5. Social Heat, comes with extensible API. You can access your data via the robust API platform provided by Social Heat. Probably other developers can create applications which can consume your data and give a much more detailed report to you. - Inspired by Google Analytics API and applications built on top of it.

Now, deploying a system for general public usage is far from near, moreover I neither have enough infrastructure or capital to create one. So, I'm on the hunt for Angel Investor for this adventure. If you are interested and like to get along, feel free to contact me on ashwanthkumar@googlemail.com. If you think Social Heat, is missing some features - feel free to suggest them. We'll try to incorporate it.

Watch this space for, Social Heat is fast approaching for closed public testing soon.