DCortesi . blog » Twitter Stats

December 27, 2007

Twitter Stats

Final Update to This Post For those not following along at home, I finally took Twitter Stats to the next level and released a webified version over at TweetStats.com. I, somewhat unfortunately, had to go with TweetStats as twitterstats.com was already taken. I made a post about it here and you can see an example of my stats on the site on the graphs page.

So I’ve been a user on Twitter for a little over a year, but it wasn’t until recently when I hit 2000 tweets that I wanted to see what my Twitter history looked like over that period. Ever being the statistics nerd, I pulled down all of my tweets and using a combination of curl, sed, grep, Excel, and Numbers, managed to generate some nice graphs.

Being the automation weenie that I am, I eventually hacked together a perl script that did everything except paste the data into Numbers.~~Although I won’t post it here (because I think the Twitterocracy would have a cow with how it’s implemented), you can DM me your email and I’ll send you the code and instructions.~~ See below - bugs be damned, I’ve made it publicly available.

Basically, the script pulls down all your tweets and stores them in an csv file. It then runs some statistics on the csv file and then copies the resulting stats to the OS X clipboard to paste into each table within Numbers. If run with a pre-existing tweets csv file, the script will calculate the different between your current status count and only download the pages necessary, thus saving the Twitter servers from some bandwidth.

For the record, here are mine.

Update Thanks to kosmar for pointing out that I can actually do this entire thing without your password. Head => Wall. I’ll be updating it accordingly and hopefully even making a webservice out of it soon.

In addition to not needing your password, the script should also adjust the times for your tweets to whatever the system time is where you run the script.

Another update: I’ve also posted the script on my site and you can download it here: twitter_stats.zip. Feel free to contact me with any questions via twitter or web.dpc at dcortesi . com.

Many people have noticed a large after-lunch spike around 2pm. At least for me, this was due to Twitter being down most of the morning one day and then tweeting like crazy when it came back online.

BUG FOUND AND SQUASHED

There was a small bug that cropped up after I switched the script to not require your password. It accounts for the odd “[Tuesday|January|2pm] Peak” that people were seeing. This bug has now been fixed and an updated script is available. My apologies.

Unfortunately, if want the most accurate tweets, you will have to rm your csv file and run the script again.

Date::Calc aka failure on line 13
Some of you (on Tiger?) may be missing the Date::Calc module that I use to figure out weekdays. Although I tried to use as few perl modules as possible, this one was essential. Use the following command (thanks to a couple twitter peeps for the reference) to install:
sudo perl -MCPAN -e 'install Date::Calc' and keep hitting ‘y’.

Final (hopefully) Update on this page as it’s getting messy.

For those of you not on OS X with Numbers, there are a few options:
@bck webified my code (w00t): Twitter Stats
@mmc decided to use gnuplot: Twitter Stats in SVG Using GNUPlot
@cbarrett modified it to utilize the Google Chart API: Twitter Stats with GChart
@kejadlen reverse engineered my original script to Ruby: Twitter Stats in Ruby

I still want to write my own webified version (Google Chart aesthetics leave a little bit to be desired…), but I have yet to settle on an option that I like.

89 Comments so far
Leave a comment

That is just too awesome.

Hi Damon,

That’s just awesome! Since I just reached the 7,000 updates score today, I’m eager to have a look at my own stats. Would you be so kind to share your scripts with me?

TIA,
Christophe LaueR

–
http://twitter.com/clauer/

O_o i’d love to see this. you got my mail …

congrat for this…can i have mine?…txs

Twitter statistics…

After I learned via @twtooltrack that Damon Cortesi had hacked a perl script to find out some basic stats of your own twitter behaviour, I just asked him for the file and ran it on my account. Here is what came out:

One more click for the large vie…

This is going to be awesome. Can I have the instructions? I am super excited to see some stats on my tweets Thanks.

[…] day to post some graphs of my Twitter usage. A huge amount of thanks goes to Damon Cortesi for his work on the Perl script that creates these stats and the Numbers template for creating the pretty […]

Slick. Thanks for sharing.

Too bad-ass! Can’t wait to unzip it! :^)

I really like using it, fun to see all the results and such. I’ll write an entry about it soon. I’ll help to promote it.

Very cool and easy to use. Thanks.

[…] stats! Thanks to Damon Cortesi (https://twitter.com/dacort) for his awesome Twitter Stats script (http://dcortesi.com/2007/12/27/twitter-stats/). And to think. I said I wouldn’t use Twitter. I said I wouldn’t become addicted. I […]

[…] was kind enough to gather my Twitter stats for me. Check out his blog for the script he created to do […]

So, I noticed you wrote that you no longer have to use your password for this script. Convenient, yes, although I just wanted to mention that for those with protected tweets it doesn’t work. Easy for me to solve. I just unprotected mine as it downloaded them, then protected again. But I think it would be handy to keep a version of this that asks for a password to get around that.

Awesome work!

Rob - good catch, I hadn’t considered that.

I’ll make one available at some point that can use your password.

[…] djmartian wrote an interesting post today onHere’s a quick excerpt […]

[…] stats Use this cute perl script + Numbers template to take a slightly different look at twitter activity. Here’s mine; I am @bkerr on […]

Does the bug you just fixed explain why DaringFireball.net and RandsInRepose.com both report post-lunch spikes?

[…] Damon Cortesi has published a nifty Perl script that generates stats based on your Twitter usage. […]

@nickon It would appear that way, unfortunately. :-\

I’m also aware some ppl don’t have Date::Calc installed and am looking for specific steps to get that installed. (Somebody else mentioned there were several steps, but I already have it installed as well as several different versions of perl :-\)

This is great! Found a typo:

“Press enter to download remaining tweets and calcuate stats!”

“calcuate” should be “calculate”

Also, for protected tweets, this would be a great use of OAuth!

http://twitter.com/oauth
http://oauth.googlecode.com/svn/code/perl/

Dude! THis is a damn sweet little perl script. We should tie it into a graphic lib so that you can have this automatically appear on your website

@whurley That’s the plan. Somebody already suggested Flot and Google Chart API (of course). It’s just a matter of sitting down and whipping up that code.

Very cool, I’m building the new http://whurley.com in perl so this will fit right in

[…] DCortesi . blog » Twitter Stats A perl script for OS X that pulls down your public tweets and allows you to graph them using Numbers… […]

Regarding implementation with Google Chart API: I’ve done it for you, for reasons I can’t entirely explain. (I was going to use tonight to start blogging again. So much for that idea.)

Right now it only emits the GCAPI URLs to STDOUT rather than actually doing anything useful like creating a web page with them. I’m going to fix that, but I’m posting this now as it’s already well past midnight. There are also a few bugfixes in there - it may be worth diffing to find them.

Ahem, not that URL. This one.

(This is why I should just go to bed now instead of continuing to hack on this.)

@Yoz Sweet, nice job! I’ll have to try it once I’ve had a little sleep too. ;o)

[…] year of Twitter. Late last night I saw a tweet from Tim Lauer referring me to a Perl script from Damon Cortesi “that pulls down your public tweets and allows you to graph them using Numbers.” Pretty […]

[…] on the lookout for analytics to describe behavior, I ran across a perl script by Damon Cortesi tonight that pulled in my Twitter feed and generated some pretty graphs. The results - my Twitter […]

You can get that CPAN install with even fewer keystrokes:

$ sudo cpan Date::Calc

[…] So here are my Twitter stats. Generated and made pretty by the awesome script created by Damon Cortesi. […]

Damon, just a few questions. I assume most of this info is global according to your list of tweeps? Because according to the stats, I’m tweeting while I sleep, and I can’t recall 300+ tweets I made on Monday. This could also be indicative of a serious addiction on my end. Is there a way to just find out my personal stats?

Could you provide a summary for each table to help us understand what each represents? For instance, I’m not clear on the difference between top conversational @’s and top @replies.

Thanks for this script, BTW. It’s very cool to see Twitter, Numbers, and Terminal merge usefully.

I tried it few weeks ago too, to determine the time of the day my friends were twitting, here is the result:

http://blog.alexgirard.com/2007/11/12/ruby-twittline/

[…] Twitter lovers out there, like me, I’ve put together a little script based on the work found here to calculate some statistics from your Twitter timeline and present them using the Google Charts […]

Hi Damon, for some reason the script hangs forever on my tweets, and there are only 32. Same with http://bradkellett.com/twitter_stats.html which is based on it. Try to crunch my Twitter at http://twitter.com/pdwn

Bonne année, best wishes for 2008.

[…] ajutorul motorului de la Google Charts. Codul a fost adaptat de autorul Brad Kallet dupa cel de la DCortesi. Dupa ce desenele sunt generate, datele personale ale utilizatorului de la Twitter sunt […]

[…] El servicio, simple como hay pocos, solo requiere que introduzcas el nombre de usuario de Twitter, tipo @tunick y te muestra gráficos. El sistema está elaborado a raíz del script para Mac de DCortesi. […]

on the Date::Calc install i am getting an error in Terminal:
Recursive dependency detected: Date::Calc => S/ST/STBEY/Date-Calc-5.4.tar.gz => Bit::Vector => S/ST/STBEY/Bit-Vector-6.4.tar.gz => Carp::Clan => J/JJ/JJORE/Carp-Clan-5.9.tar.gz => Object::Deadly => J/JJ/JJORE/Object-Deadly-0.09.tar.gz => Carp::Clan. Cannot continue.

anyone else getting this?

[…] a bit like navel gazing to me. So by that logic Twitter would be Micro Navel Gazing (TM). Enter Twitter Stats, The Atomic Navel […]

[…] todo gracias a Brad Kallet, quien adaptó el script que Damon Cortesi realizó para la plataforma Mac OS X, de manera que a través de su web podemos recibir una serie […]

I thought I was never gonna be able to do this, not knowing any Perl and being a terminal newbie, but it was easy enough.

I wrote a quick guide how to it: http://www.wolfslittlestore.be/twitter/how-to-get-your-twitter-stats-for-2007

Kyleon: Yes, I got that. I solved it by updating Bundle::CPAN first. The command would be as follows:

sudo perl -MCPAN -e ‘install Bundle::CPAN’

As usual, keep hitting y to install all the dependencies. Then go back and re-install Date::Calc.

Twitter stats using Ruby…

I saw Damon Cortesis Twitter Stats script last night, and decided to make a Ruby version….

[…] My Twitter Stats of 2007 thanks to Damon Cortesi. […]

Ooh, that looks so cool! If only I could run it, though; I have neither Perl nor OS X.

[…] Cortesi has written a great little perl script that will generate your Twitter Stats which are then turned into some cool graphs of using the Numbers […]

[…] a Numbers template to paste them into. Since I don’t have Numbers, I’ve modified his script to write a file that can be read by gnuplot, and wrote a basic gnuplot script to output an SVG file […]

[…] es posible gracias a Brad Kallet, quien adaptó el script que Damon Cortesi realizó para la plataforma Mac OS X, y que a través de su web se puede generar una serie de […]

[…] so pretty Twitter Stats With the recent surfacing of Damon Cortesi’s Twitter Stats I experienced my first real case of Mac envy. I tried running the Perl script on my Debian box, but […]

[…] Download the Twitter stats zip at http://dcortesi.com/2007/12/27/twitter-stats/ . […]

Hi Man.
Just a question. Why did you decide distribute your software for local running instead of hosting it in anyserver?
Is it due to the Twitter’s API usage restrictions?
Thanks in advance.

Hey Xurde - Perl was the easiest thing for me to hack the script up in and I really liked the way that the graphs generated by Numbers looked, so that’s the main reason I didn’t webify it. I didn’t initially expect to distribute it either, but I really enjoyed hacking away at it.

I’ve found a couple graphing packages that I like, but just have to sit down and pound the code out to support one of them.

Local development was a much quicker way for me to utilize currently existing tools to achieve a more pleasing result.

[…] care to know your Twitter Stats? Posted in twitter, stats | Trackback | del.icio.us | Top Of […]

[…] Get stats on your Twitter usage (tags: analytics twitter) […]

[…] DCortesi . blog » Twitter Stats A Perl script that pulls data from your public Twitter stats and produces data to be pasted into a Numbers spreadsheet for data visualization… (tags: twitter visualization osx perl) […]

[…] DCortesi . blog » Twitter Stats (tags: twitter statistics script osx) […]

Doesn’t work with foreign charaters like the german umlauts (ä, ö, ü) if you wanna test it, run the script against my username: «daskaja». Seems like sed is the problem.

[…] to Damon for the hard […]

[…] habla hoy sobre Twitter Stats1, un programa escrito en Perl que los usuarios de Mac OS X que tengan Numbers instalado pueden […]

[…] Twitter Stats. Basically, the script pulls down all your tweets and stores them in an csv file. It then runs some statistics on the csv file and then copies the resulting stats to the OS X clipboard to paste into each table within Numbers. © 1999-2008 Justin Blanton (e-mail) e v e r y t h i n g i s r e l a t i v e In partnership with […]

Amazing script, thank you

[…] Jens Meiert via Twitter bin ich heute mal auf Twitter Stats gestoßen, einem Skript von Damon […]

[…] TwitterStats […]

[…] You can get the script over at http://dcortesi.com/2007/12/27/twitter-stats/ […]

[…] fem måneder siden jeg skrev min første Twitter-status, og for noen dager siden kom jeg over Twitter Stats av Damon Cortesi som kartlegger Twitter-bruken […]

[…] thing I’m lazy… wait a day or two after seeing the original code and someone makes a nice little Twitter stats web service out of it (found via a CogDog […]

[…] when I saw this Twitter stats script, I was intrigued to see what my numbers looked like. Call it internet naval-gazing, but I find it […]

[…] Cortesi wrote a cool perl script that sucks down a user’s tweets from 2007 and creates pretty charts via a Numbers.app […]

我的twitter统计…

下面是来自这里的我的twitter使用情况的统计图表： Current statuses count: 4124 —- Tweets per Hour ——– Tweets per Day ——– Tweets per Month ——– Top @replies ——– Top overall @’s —- 这里有相关的介…

[…] January 7, 2008 · No Comments Thanks to a random discovery on Tara Hunt’s Flickr stream, I found out about Damon Cortesi’s script for generating charts out of Twitter. […]

[…] up in the middle of the year thanks to programs like Twitterrific and Twitterberry. Now thanks to Damon Cortesi, who hacked up a quick and dirt script, you can see a visualization of my Twitter activity for 2007 […]

[…] cool. He also mentions he’s wanting to get his up on the web too. Quick reference to his original perl script. Tags: Twitter, Twitter Hacks, Twitter Stats, Perl, Ruby SHARETHIS.addEntry({ title: “Twitter […]

[…] not jealous, but insane? Totally. No really, I thought “hey, that’s cool.” The script he linked to was for OS X, which I don’t use so I waited. Today I found a site that did it online. The […]

[…] DCortesi . blog » Twitter Stats graph your twitter stats (tags: twitter statistics stats visualization script analytics web2.0 chart graph) […]

[…] week ago, Damon Cortesi (who happens to be one rockin’ guy) wrote a little Perl script to help Twitter users create some really pretty graphs that chart their Twitter usage over the past year. In the week or so that has followed, everyone […]

[…] Cortesi over at DCortesi Blog recently commented about tracking his Twitter Stats. With over 2,000 Tweets (sum of Twitters), he wanted to understand what his history looked like […]

I am new to twitter (18 posts), but when I tested the script it either took over 10 minutes and revved up my macbook fan because it was working like crazy… or it just hung. I have the newest version of leopard, can that be the culprit?

[…] they should come out with social stats, like the ones Damon Cortesi and a few others made. But of course with stats being a native part of the platform it would be […]

[…] Stats01.30.08 | Comment? Last month a twitter stats script was released by Damon Cortesi. The small script connects to twitter and downloads all of your tweets for statistical analysis. […]

Thanks for the great script! I just posted stats to my blog. I’m running Leopard and was not only missing the Date::Calc module. I also was missing LWP which posed difficulty in downloading bundles. Luckily, I am a user of MacPorts and after installing lynx and wget (also surprisingly missing) I was able to reset the CPAN setup and proceed. Hopefully my experience will help someone else.

[…] in the month, we told you about TwitterStats, a downloadable script that would graph your Twitter statistics. The author of TwitterStats, Damon […]

[…] que utiliza esta medio. Ya existen APIs para jugar con nuestras estadísticas de microbloggeo como TwitterStats, sitios que pretenden acercarnos a otros usuarios que comparten intereses más o menos parecidos, […]

[…] TweetStat […]

[…] fem måneder siden jeg skrev min første Twitter-status, og for noen dager siden kom jeg over Twitter Stats av Damon Cortesi som kartlegger Twitter-bruken […]

[…] TwitterStat Twitter统计数据 […]

[…] some ways to get this information and stumbled across http://tweetstats.com, which is based on a perl script to generate some cool information but it takes too long and is not exactly what I wanted. At […]

[…] Cortesi wrote a pretty nifty script to grab some Twitter statistics. A bunch of people modified his script in order to webify it, plot the results using gnuplot, and […]

RSS feed for comments on this post. TrackBack URI

Leave a comment
Line and paragraph breaks automatic, e-mail address never displayed, HTML allowed: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <code> <em> <i> <strike> <strong>

Coding, Security, and maybe a little bit about Damon Cortesi

mdy: This is fantastic stuff, Damon!

Thanks for such amazingly quick turn around on those suggestions! *thumbs up*

Damon: Version 0.2.6 is out, and it displays how many messages you’ve deleted.

Damon: @mdy Thanks for your great, detailed feedback! The first two requests have been added in version 0.2.5.

mdy: Hi Damon! Many thanks for this bookmarklet. It works great on my Firefox 2.0.0.14 on WinXP.

Some feedback:...

Damon: @Jon That is definitely a valid statistic and is a common indicator “most of the time”. There...

Dining (rss) (1)
germany (rss) (1)
Personal (rss) (347)
- Chi-town (rss) (74)
- PacWest (rss) (55)
Photography (rss) (6)
Reviews (rss) (2)
Site (rss) (710)
- Imported (rss) (693)
Tech (rss) (249)
- Coding (rss) (37)
- Internet (rss) (47)
- OS X (rss) (7)
- Security (rss) (62)
Travel (rss) (9)

Andy
claudius
CTA Tattler
Fabulous Adventures In Coding
Jeremy Zawodny’s blog
Kasia
Leyrer
Life on the road
Likert
MSDN Weblogs
Rnast
Schneier on Security
Vitaly

December 27, 2007

Twitter Stats

Recent Comments

Ads (Beta)

Categories

Archives

Search

Blogroll

Latest Mobile Pics