DCortesi . blog

Every once in a while I revisit Twitter Spam. It’s always interesting to see the evolution of spammers as it’s happened on other social networks before and their behavior on Twitter is similar.

Essentially, they get smarter. It’s as simple as that. Early spammers on Twitter would simply follow lots of people and send obviously spammy messages. Twitter put a cap on that with their follow limits. Spammers of course then gamed the system by figuring out who had auto-follow turned on and following those folks as well as following regular people and unfollowing when not mutual. Twitter is since in the process of disabling auto-follow, though other services exist and will pop up.

The latest iteration, which I’d seen evidence of before but only from the initial prep stage, is the “almost real” accounts. Let’s take a look at @james_mahoney.

Spam(?) Account on Twitter

Now this looks like a fairly normal account. Tweeting about basketball, college. But let’s take a closer look at a couple of those messages. Specifically because they might look a bit familiar…

Yup, that’s right. Those tweets are simply duplicates of tweets made earlier by other, real people. This account is simply duplicating them to appear legitimate. Now those links for the Kindle are starting to look a little suspicious, too. If we take a closer look, we notice that it’s a legitimate link to Amazon, but with the spammer’s associate code in the URL.

You can see that his account also looks real enough that people even engage in conversation.

So the associate code means this spammer will get a few bucks if somebody actually buys the Kindle. How many times does @James_Mahoney send out these tweets? About 200 out of 800 - just see for yourself. Even a few hits makes it worth it if this is automated in any fashion.

A final note - want to find some other accounts in the initial phase of this scam? Just click through some of the recent follower icons on @James_Mahoney’s page, particularly the ones with the default avatar - you’ll notice they all have an eerily similar set of first messages. Welcome to Social Network Spam.

Thanks to Mike Dahn for the heads up on Mr. Spammy Spammer.

(p.s. Don’t forget to click my affiliate link to the right. ;))

Tweetie has become my exclusive application on the iPhone for Twitter usage. Between multiple account support, saved searches, and a quick, simple interface it replaced the few other apps I had to use simultaneously to achieve all of these.

As part of the Twitter application stats I maintain over at TweetStats, I would notice Tweetie usage increasing on the weekends. My assumption is that mobile use rises on the weekends as people are not on their work computers.

As part of a potential new service offering at TweetStats (activity graphs for your app!), I decided to verify my assumption*. As you can see, there’s a definite increase in usage of Tweetie on the weekends (gray bars).

Just for comparison sake, let’s take a look at another popular mobile app, TwitterFon. We see the same characteristics, with surprisingly similar trends.

To hit the point home, here’s a graph showing TweetDeck usage (the most popular desktop client) where we see a slight decline in usage during the weekend over the prior few days. However, usage is still comparable to the beginning of the work week.

And finally, a brief comparison of Tweetie and TweetDeck and their respective usage.

Mobile and Desktop application usage over the course of two weeks

*These statistics are generated using data collected at TweetStats utilizing the fantastic Gnip service and represent the large majority of updates posted on Twitter. I do not collect statistics on protected updates.

I just realized I never posted here about my recent Twitter Bio Search Tool, TweepSearch. I guess I’ve just been a little heads down lately. I’ve been busy with various Twitter apps, work, and I’ll be heading to Boston tomorrow for the SOURCE con and to visit with friends and family.

Anyway, TweepSearch - the original idea of the site was to allow somebody on Twitter to search the bios of their followers. It was inspired by a tweet from @SethSimonds and you can read more on the About page. However, once I started building it, I realized I was creating a more generic Twitter bio search application. I just updated the application yesterday and it now allows you to login (non-SSL, I’ll be fixing that eventually) and (un)follow directly from the interface, searches all Twitter profile fields by default, and allows you to search your friends and followers. Some examples:

Search for security peeps in Seattle: location:seattle security
Search my friends and followers for security peeps: @dacort security
Search my friends for peeps in Boston: @dacort only:friends location:boston
Except for those folks I’ll see at the con ;): @dacort only:friends location:boston -hacker -security

As you can see, the search syntax is pretty extensive. I’d like to add geo-based searches in the future as the full-text indexing engine I’m using supports it. Thinking Sphinx, the Rails plugin for Sphinx is also amazing and I have to thank EC2 for allowing me to scale so quickly when the site first got hammered due to a great post on louisgray.com courtesy of Jesse Stay.

I’m currently just over 1.4 million Twitter profiles indexed and constantly growing.

As part of a recent project, I’ve been digging into some pretty cool data using Tableau. One of the instant deciders somebody on Twitter makes when they’re followed by a new user is their friends/followers ratio. If a user has lots of friends, but few followers, they’re not likely to be very interesting or can even be spammy accounts.

As part of their attempt to combat spam, Twitter initially limits the number of people you can follow to 2,000. Once you have been vetted by other users in the form of them following you, you can add more friends. This creates an interesting distribution when you start analyzing the friends to followers count. Taking a look at the image below, there are several things to note.

• There’s a large majority of Twitter users within the initial friend/following block of 2,000
• People rarely have over 1,000 friends without at least 250 people following them back
• You can obviously see that Twitter allows you to start adding more friends once you’ve hit 1,800 followers
• Once that limit has been passed, people generally continue to have a fairly steady ratio of 1:1
• However, there are a fair number of users who begin to restrict their # of friends after that point, but continue to receive more followers once they’ve been “acknowledged”
• Most of the users with more friends than followers in the bottom right are early Twitter accounts before Twitter imposed their limit
• There also seems to be a significant group of celebrity or otherwise popular users that have limited friends, but stretch up the left side with a large number of followers

What other conclusions do you draw from this? There are some other interesting behaviors once you dive into the 2k section.

I get very bent out-of-shape when people post inaccurate or misleading statistics. TechCrunch just recently had a post on the Top 20 Twitter Applications in which they used traffic to the applications’ web sites to determine the top 20 apps. While they admit it’s not the best, they left out one of the top Twitter clients (which didn’t make Loic happy, of course) and the data really is not representative of the truth.

As part of TweetStats, I pull in data from Gnip on every single Twitter update and the associated application. So, here are the real Top 20 Twitter applications.

Footnote: This is out of slightly over 50 million Twitter updates in January. It does not included protected users and there may be some tweets missing due to downtime in Twitter or Gnip. But for the most part, this is very representative.

Update: Another quick stat - in January, 1,231 different client applications were used to post updates to Twitter.

Approximately 17 hours ago, I received a tweet from @KymPossible regarding an app to pull your Twitpics out of your timeline. Apparently, @donttrythis wanted to pull all of his TwitPics out of his Twitter timeline. Always one for a challenge and with a fond love of one-liners, I whipped up this quick hack that pulls TwitPics out of your Twitter timeline and sticks them in an HTML file.

It’s ugly, it can be factored, it’s invalid HTML and the API barfs more often than returning the correct data…but it works. For the sake of sharing, here’s the little “script”.

That will download the last 750 updates (the Twitter API barfed over that number usually), run some shell-fu (this is where the factoring would come in), retrieve the twitpic images and sort them in chronological order. It should work on most UNIX-based systems, but I wrote it on OS X. Have a nice day.

(I really need to make a new post on my other new Twitter tool that can be used to search Twitter bios. I’ll get to that soon…)

p.s. I love Gist.

For those wanting to wipe out DM’s on Twitter after the recent phishing issue, feel free to make use of the DM Whacker.

Depending on how little sleep I want to get, I will add functionality in the near future Functionality has been added to delete tweets with certain text in them. For the geeks out there, regular expressions are even supported!

Please leave any comments on the original blog post.

Christmas day, as usual, has been a busy and productive day for me. I seem to be getting in the habit of coding up random stuff when I get a week to myself, and Christmas 2008 was no exception. I was sitting around this evening working on a project, when I got side-tracked…and then side-tracked again. I realized that through a (intentional?) quirk of how Twitter stores it’s relationship data, the first person you ever followed could be determined very easily. A few hours later and I tossed up My First Follow, an application that shows who your first friend on Twitter was. Ah, memories.

You’ll notice I also updated my blog design. Much more copacetic, in my opinion.

I hope everybody had a good Christmas, I sure enjoyed mine.

In other news, I started a new site today called Startup Security. You can read the Welcome post, but I mainly created the site because I’ve come across several security issues at startups in the past few months and I want to spread awareness about integrating security into the development life-cycle early on.

In coming days and weeks, I’ll post about different vulnerabilities I’ve come across as well as things to look for if you’re a startup and aren’t familiar with security. The point I’m trying to get across is that security doesn’t have to be a $50,000 investment to be effective. Heck, I could probably spend half-a-day talking to devs and know whether or not they understand security enough to not build a reasonably secure site…

In any case, Check it out at StartupSecurity.info. I hope it becomes a useful resource.

Last night, around 6pm EST, I started asking around Twitter to see what people were doing to help out with Gustav efforts. What I found were a lot of links to how people could find information about Gustav, but not very much in the way of active efforts to provide a means for disseminating actual requests for help.

As an example, Twitter is a pretty big echo chamber and a simple request can get lost, even in targeted searches. As I couldn’t find anything else and my cohort was having trouble finding an official means of SMS communication via the Red Cross, GustavTracker was born.

I don’t know if this is the right way to address this problem, but I’m trying to do what little my nimble keyboard fingers can. This morning, I also came across a Gustav Information Center Wiki and it seems they’re doing some similar work. I’m currently in the process of trying to collaborate with them.

In the end, I fear it’s simply too late to get the necessary information made available. However, if you would like to help, please pass on the information about GustavTracker for those in need of something that the web community might be able to assist with.

It should be noted that the Red Cross has the @safeandwell Twitter account that ties in to the official Red Cross Safe & Well List, but trying to find information relative to this channel was an exercise in futility.