Damon Cortesi's Blog

Well, I’m about 45 minutes late on this, but I did want to mention that two years ago on December 27, 2007, I released the first version of my Twitter Stats script.

It was a hack. It scraped the twitter.com website. It was in perl. It exported into your clipboard. And the data had to be pasted into a Numbers template. But it worked. And it was the beginning of a long journey that is culminating in something I’m going to announce tomorrow/today.

On this anniversary, I also just made an update to TweetStats that allows you to click on any month in your tweet timeline and zoom in to see your stats on that month. As an example, here are my Twitter stats for December 2007. Ah, memories. And speaking of memories, a little-known feature of the TweetCloud is that you can click on any term and it will search your tweets for that term.

Reminisce++

Dear Twitter, I urge you to please hire a CSO. A Chief Security Officer. Somebody to lead the charge and organize a security team around what is arguably one of the biggest things to happen to social media in the past 10 years. A security team to balance the risk of being completely open with our lives, happily geo-tagging our way to a billion-dollar valuation of Twitter.com. A security team to realize that we are quickly losing any privacy we have had by opting in to the great and amazing features that you’re releasing on a daily, if not weekly, basis. A security team to protect us from our own poor choices.

There’s a story on TechCrunch that Twitter was hacked tonight by the Iranian Cyber Army. Regardless of whether this is true or not, Twitter desperately needs an individual in their organization to guide them on security as they carve the path in both social media and the openness and revealing of privacy on the web. From spam to meter-precision geo-location, you are failing in this respect.

Over the course of the past year, I’ve alerted Twitter to a number of different security incidents. From cross-site-scripting to server mis-configurations to a simple heads-up about other security issues I’ve seen randomly crop up. The problem is, the same issues continue to crop up on a regular basis. Sadly, for a company with $150 million dollars invested at a $1 billion valuation and over 100 employees, they have no Chief Security Officer. And I’m not even sure they have dedicated security engineers. Just rockstar developers.

To Ev Williams, CEO of Twitter. And Dick Costolo, COO of Twitter. I beg of you. Make the investment in security before It’s too late. Twitter needs to be aware and proactive about security if it is to continue in the way that you dream of. Security is not something that can be solved as easily as bringing in a firm to do a two-week assessment and call it good. It’s a culture that has to be built from the inside that permeates not only to those responsible for systems and code, but also those that are simply part of the company in any way shape or fashion.

While no organization is ever completely secure, it is critical at your stage that you start building security from within the organization, instead of having it beat upon you from painful experiences. This is a lesson that it took Microsoft many years and millions of dollars to learn and one that you (Twitter) should proactively attack.

Please. I understand you’re building some awesome business intelligence and some advertising that we’re just really going to love. But realize that you are changing the way we share data on the Internet. And not only do you need to be the leader in social media and openness, you need to be the leader in social media privacy and security.

After posting on 2009 airline activity on Twitter, I got a couple requests that it’d be interesting to see activity from coffee brands on Twitter. (Did I mention that I live in Seattle, home of the illustrious @Starbucks? )

A few tweaks and 77 minutes later, I was able to put together the following graph of tweets from each coffee brand’s Twitter account from January to October 10 of this year.

A few interesting observations. There were only four brands even on Twitter in January of this year. Starbucks and Dunkin Donuts dominated then, as they do now. There was an interesting peak among most brands around July and September of this year, with many having their activity taper off after that.

Dunkin Donuts has been tweeting a lot this month. If we take a closer look at their Twitter stats from October, we see that October 6th was a busy day. A little research shows that the 6th was the day they came out with their annual fall lineup and that they also had a promotion for firefighters in DC that day.

Amazing what a little visualization will show.

I’m mildly amused at the response of Pepsi and the negative response to their “AMP UP Before you Score” iPhone app that objectifies women and parodies off the typical male approach to dating in the year 2009.

Conversations flared on Twitter this weekend after the company launched their iPhone application. Interestingly enough, the response from AMP and Pepsi at the moment is to try to cram a PR response into a 140 character tweet.

Originally tweeted out from the @AMPwhatsnext account, the @Pepsi account re-tweeted the message shortly thereafter. At the time of this writing, this is the only official response I could find. Nothing on the Pepsi website or the AMP website. Just an apology crammed into 140 characters using numbers to abbreviate words.

Is this really what PR has come to in the world of Twitter? The other side of this, though, is the interesting fact that this may be all that’s needed to address this bit of PR. Where opinions can blow up in a matter of minutes and spread like wildfire, the days of delicate PR are long gone for real-time PR in the nature of the medium.

This has certainly been an interesting year for little ‘ol Twitter. Growth has exploded, celebrities have been joining in droves and Twitter continues to expand their feature set in an amazing effort to make those 140-character tidbits all the more valuable. Brands have also noticed the value in Twitter, listening in on the thoughts of millions of people in hopes of not only improving customer satisfaction, but winning customers over with a personal touch. Fellow Twitterer Dave Peck experienced this earlier this year when Southwest Airlines tried to help him out after getting stuck in Austin.

How appropriate, then, is the graph below that shows @SouthwestAir as the most active airline on Twitter, based on the number of monthly tweets from January to September of this year.

I decided to put this graph together after reading @BrianSolis‘ post on airline activity in August. Curious what the rest of the year looked like, I pulled some data from TweetStats and decided to try to represent the data in a StreamGraph, courtesy of Lee Byron’s awesome StreamGraph work. This is my first attempt and could certainly use a little tweaking, but the trends in airline activity over the course of the year are readily apparent.

In addition to simply seeing how active airlines have been over the past year, the graph also shows the overall number of tweets for each airline (font size) as well as the most active month for each airline (placement of their Twitter username). There are some airlines not marked on the graph as their activity is insubstantial.

Also of note is @FlyHawaiian, whose usage of Twitter increased tremendously in September.

Detailed stats for any of the airlines can, of course, be found on TweetStats.

If I’ve been a hermit lately, it’s because I’ve been hard at work in the Twitter world. Apologies to my family for not calling as often.

I’ve been busy during the day doing some fun work on a variety of things. Between the hours of midnight and 5am, though, I’ve put together a few fun new Twitter tools. The first was inspired when I spoke at the first Twitter conference back in May. I got followed by a lot of interesting people, but some of them got lost in the random follows I get as well.

So I built FollowBack y’all – a simple little tool to follow back people that recently followed you, based on their tweets. Once you log in, you’re presented with a list of recent followers you’re not following and a search box. Entering a term in the search box searches tweets and tries to match up new followers that mentioned that term. Then you can follow back with one click of a button.

The other couple tools are based off something I built that could be useful for Twitter developers – programmatic access to Twitter avatars based off a username or user id, something Twitter doesn’t provide at the moment.

The first is called Commonality – a tool to show you what friends and followers you have in common with somebody else on Twitter. Seeing as how it was built in about an hour, the interface is certainly lacking but here’s an example that shows the common friends and followers I have with @securitytwits.

The second is called FollowMinder – a tool to remind you when you followed somebody on Twitter. Again, more of a demonstration of the TwitterAvatar tool, but interesting to remind myself when or why I followed somebody. As an example, @wardspan was the second person I followed among a few good friends.

Finally, I was curious one night who I’ve blocked on Twitter and the API had just been updated with new “block” methods. A little while later and I had put up TwitBlocked – a tool see who you’ve blocked on Twitter.

I hope you enjoy!

Every once in a while I revisit Twitter Spam. It’s always interesting to see the evolution of spammers as it’s happened on other social networks before and their behavior on Twitter is similar.

Essentially, they get smarter. It’s as simple as that. Early spammers on Twitter would simply follow lots of people and send obviously spammy messages. Twitter put a cap on that with their follow limits. Spammers of course then gamed the system by figuring out who had auto-follow turned on and following those folks as well as following regular people and unfollowing when not mutual. Twitter is since in the process of disabling auto-follow, though other services exist and will pop up.

The latest iteration, which I’d seen evidence of before but only from the initial prep stage, is the “almost real” accounts. Let’s take a look at @james_mahoney.

Spam(?) Account on Twitter

Now this looks like a fairly normal account. Tweeting about basketball, college. But let’s take a closer look at a couple of those messages. Specifically because they might look a bit familiar…

Yup, that’s right. Those tweets are simply duplicates of tweets made earlier by other, real people. This account is simply duplicating them to appear legitimate. Now those links for the Kindle are starting to look a little suspicious, too. If we take a closer look, we notice that it’s a legitimate link to Amazon, but with the spammer’s associate code in the URL.

You can see that his account also looks real enough that people even engage in conversation.

So the associate code means this spammer will get a few bucks if somebody actually buys the Kindle. How many times does @James_Mahoney send out these tweets? About 200 out of 800 – just see for yourself. Even a few hits makes it worth it if this is automated in any fashion.

A final note – want to find some other accounts in the initial phase of this scam? Just click through some of the recent follower icons on @James_Mahoney’s page, particularly the ones with the default avatar – you’ll notice they all have an eerily similar set of first messages. Welcome to Social Network Spam.

Thanks to Mike Dahn for the heads up on Mr. Spammy Spammer.

(p.s. Don’t forget to click my affiliate link to the right. )

Tweetie has become my exclusive application on the iPhone for Twitter usage. Between multiple account support, saved searches, and a quick, simple interface it replaced the few other apps I had to use simultaneously to achieve all of these.

As part of the Twitter application stats I maintain over at TweetStats, I would notice Tweetie usage increasing on the weekends. My assumption is that mobile use rises on the weekends as people are not on their work computers.

As part of a potential new service offering at TweetStats (activity graphs for your app!), I decided to verify my assumption*. As you can see, there’s a definite increase in usage of Tweetie on the weekends (gray bars).

Just for comparison sake, let’s take a look at another popular mobile app, TwitterFon. We see the same characteristics, with surprisingly similar trends.

To hit the point home, here’s a graph showing TweetDeck usage (the most popular desktop client) where we see a slight decline in usage during the weekend over the prior few days. However, usage is still comparable to the beginning of the work week.

And finally, a brief comparison of Tweetie and TweetDeck and their respective usage.

Mobile and Desktop application usage over the course of two weeks

*These statistics are generated using data collected at TweetStats utilizing the fantastic Gnip service and represent the large majority of updates posted on Twitter. I do not collect statistics on protected updates.

I just realized I never posted here about my recent Twitter Bio Search Tool, TweepSearch. I guess I’ve just been a little heads down lately. I’ve been busy with various Twitter apps, work, and I’ll be heading to Boston tomorrow for the SOURCE con and to visit with friends and family.

Anyway, TweepSearch – the original idea of the site was to allow somebody on Twitter to search the bios of their followers. It was inspired by a tweet from @SethSimonds and you can read more on the About page. However, once I started building it, I realized I was creating a more generic Twitter bio search application. I just updated the application yesterday and it now allows you to login (non-SSL, I’ll be fixing that eventually) and (un)follow directly from the interface, searches all Twitter profile fields by default, and allows you to search your friends and followers. Some examples:

Search for security peeps in Seattle: location:seattle security
Search my friends and followers for security peeps: @dacort security
Search my friends for peeps in Boston: @dacort only:friends location:boston
Except for those folks I’ll see at the con : @dacort only:friends location:boston -hacker -security

As you can see, the search syntax is pretty extensive. I’d like to add geo-based searches in the future as the full-text indexing engine I’m using supports it. Thinking Sphinx, the Rails plugin for Sphinx is also amazing and I have to thank EC2 for allowing me to scale so quickly when the site first got hammered due to a great post on louisgray.com courtesy of Jesse Stay.

I’m currently just over 1.4 million Twitter profiles indexed and constantly growing.

As part of a recent project, I’ve been digging into some pretty cool data using Tableau. One of the instant deciders somebody on Twitter makes when they’re followed by a new user is their friends/followers ratio. If a user has lots of friends, but few followers, they’re not likely to be very interesting or can even be spammy accounts.

As part of their attempt to combat spam, Twitter initially limits the number of people you can follow to 2,000. Once you have been vetted by other users in the form of them following you, you can add more friends. This creates an interesting distribution when you start analyzing the friends to followers count. Taking a look at the image below, there are several things to note.

• There’s a large majority of Twitter users within the initial friend/following block of 2,000
• People rarely have over 1,000 friends without at least 250 people following them back
• You can obviously see that Twitter allows you to start adding more friends once you’ve hit 1,800 followers
• Once that limit has been passed, people generally continue to have a fairly steady ratio of 1:1
• However, there are a fair number of users who begin to restrict their # of friends after that point, but continue to receive more followers once they’ve been “acknowledged”
• Most of the users with more friends than followers in the bottom right are early Twitter accounts before Twitter imposed their limit
• There also seems to be a significant group of celebrity or otherwise popular users that have limited friends, but stretch up the left side with a large number of followers

What other conclusions do you draw from this? There are some other interesting behaviors once you dive into the 2k section.