DEV Community

Burke Holland
Burke Holland

Posted on • Edited on

​​Someone has created a twitter bot that can identify pictures of Ed Sheeran

​​A developer named Scott Cate at Microsoft has built a bot that does image recognition, and, apparently Ed Sheeran recognition as well.
​​
​​Like any bot worth it’s salt, this one runs on Twitter and can be summoned using the #vision_api hashtag. After a slight delay (presumably because a computer in a cloud somewhere is examining the photo with a monocle), a reply is sent with a verbal description of the image.

#vision_api pic.twitter.com/nblqbhI7NA

— Scott Cate, Seattle #MSBuild (@scottcate) April 3, 2018

Confidence: 91.07 %

Vision API: a sofa in a room

Full API Result: https://t.co/BzV6Jh3BUi

— #vision_api (@vision_api) April 3, 2018

​​
​​The bot is pretty accurate as far as artificial intelligence goes. That monocle is doing it’s job. Most real-world objects seem to be readily identifiable. And of course it wouldn’t be a proper technological achievement without being able to correctly distinguish hot dogs from…..everything else.

Another use of #vision_api pic.twitter.com/wmPMDFW6lD

— Scott Cate, Seattle #MSBuild (@scottcate) April 4, 2018

Confidence: 96.11 %

Caption: a hot dog on a bun

Tags: hot, dog, food, dish, sandwich, bun, hotdog, paper, fries, sitting, bread, plate, long, table, covered

Full API Result: https://t.co/0GVsvX0Fjy

— #vision_api (@vision_api) April 4, 2018

​​
​​People have been tagging their photos on Twitter for the bot, and the internet does not disappoint. If you’re going to be a bot on Twitter, you can’t half-ass it.

#vision_api pic.twitter.com/nV2f42glgP

— Gabriel Robert (@kvlt_grobert) March 30, 2018

Confidence: 50.75 %

Vision API: a group of toy people

Full API Result: https://t.co/G3B4VKNEvw

— #vision_api (@vision_api) March 30, 2018

​​
​​Not quite, bot, but good guess. I suppose “Cow Onesie” is a tad weird. What will the humans do next!?!
​​

Hey @vision_api what is this?

My 5 year old said it's an Easter bunny dog. 🤣 pic.twitter.com/VdO7qZL0Zs

— René Schulte @ #MSBuild (@rschu) April 2, 2018
​​

Confidence: 24.62 %

Vision API: a close up of a bird

Full API Result: https://t.co/8ASKa051PU

— #vision_api (@vision_api) April 2, 2018

​​​​Uh, no. But to be fair, I have no idea what that is either. Is that supposed to be a rabbit? That is honestly terrifying.
​​
​​I happen to notice the trend and tagged a picture of a cheeky Ed Sheeran. I figured the bot would know that it was a man, but I did not expect that the bot would know WHICH man.

mom said you had to let me use the xbox pic.twitter.com/jwBf2AEf1Q

— dallas (@mixedhunty) April 1, 2018

Confidence: 86.74 %

Vision API: Ed Sheeran standing in a room

Full API Result: https://t.co/cjZrR3WvWv

— #vision_api (@vision_api) April 4, 2018

​​​​THAT’S ED SHEERAN. Even the machines know. You can’t be artificially intelligent if you can’t identify the talent who once crooned out, "And I know you love Shrek / Because we've watched it 12 times."
​​

OMG! ED SHEERAN, AM I RIGHT? 😍 pic.twitter.com/M7D6FyDqYs

— Bëłłä Κ Łø:śt (RT PINNED) (@Azzurioak) April 10, 2018

​​​​You are SO right!
​​

Confidence: 93.70 %

Caption: Ed Sheeran wearing a blue shirt

Tags: person, man, clothing, shirt, looking, sweater, holding, front, wearing, standing, hand, black, blue, using, young, phone

Full API Result: https://t.co/tQX3medXjX

— #vision_api (@vision_api) April 11, 2018

​​
​​Not to mention he’s “wearing, standing” and “holding”. You look great no matter what you do, Ed!
​​
​​As it turns out, the bot is not mutually exclusive to Ed. It can recognize most celebrities. Even the ones that you probably wish it wouldn’t.

Martin Shkreli investor to recover losses https://t.co/ZRTK5O4yQs pic.twitter.com/4KyPg4m1HT

— Former SEC Branch Chief (@LisaBraganca) April 11, 2018

Confidence: 84.31 %

Caption: a man standing in front of Martin Shkreli, Benjamin Brafman posing for the camera

Tags: person, man, suit, standing, outdoor, posing, holding, photo

Full API Result: https://t.co/R7gB77N04M

— #vision_api (@vision_api) April 11, 2018

​​​​Although sometimes it does get confused, like when it mistook Scarlett Johansson for Elizabeth Olson in this Avengers poster.
​​

Testing #vision_api with an #Avengers #InfinityWar poster pic.twitter.com/5u57N1g6FL

— Shahed Chowdhuri @ Microsoft (@shahedc) March 29, 2018

Confidence: 31.95 %

Vision API: Elizabeth Olsen, Danai Gurira, Chris Hemsworth are posing for a picture

Full API Result: https://t.co/kgTEehykUT

— #vision_api (@vision_api) March 29, 2018

​​​​Apparently Chris Hemsworth is unmistakable, even with one eye missing. But Elizabeth Olson For Scar Jo? Come on. They aren’t exactly identica……HOLY CRAP ELIZABETH OLSON IS IN THE PICTURE…

​​It can get things pretty wrong too. It seems to have trouble with identifying “spikey bracelets”…

#vision_api pic.twitter.com/MQFdrqA5cp

— Purple Puppy🐾 BLACK CAT D.VA (@PuppyOwO) April 7, 2018

Confidence: 48.77 %

Maybe: a person sitting on a bed

Tags: person, indoor, sitting, man, looking, small, top, dog, table, bed, little, young, holding, woman, red, white

Full API Result: https://t.co/LkIqrAQlld

— #vision_api (@vision_api) April 7, 2018

​​​​Maybe. Or maybe just someone who needs to clean their room, but wants to look really good doing it.
​​
​​And it really has a problem with furries. Not an ideological problem, but definitely a problem.

・ 。
☆∴。 
 ・゚
。★・
  ・ ゚。   *
  ・ ゚
。・゚★。
   ☆゚・。°. ゚
  ゚。·・。 ゚
   ゚ .。☆。★ ・
  
☆ 。・゚*.。
    * ★ ゚・。 * 。
    ・  ゚☆ 。 pic.twitter.com/i9YUwZWFeK

— SwiftOnSecurity (@SwiftOnSecurity) April 7, 2018

Confidence: 46.07 %

Maybe: a dog sitting on a bicycle

Tags: bicycle, dog, sitting, black, cat, white, standing, brown, large, man

Full API Result: https://t.co/igqN0xGoSr

— #vision_api (@vision_api) April 7, 2018

​​This is all made possible by Microsoft’s Computer Vision API, which says that it can “Extract rich information from images to categorize and process visual data.” Aside from being able to recognize celebrities and pharma bro’s, it can also recognize handwriting in images and popular landmarks as well. Microsoft says that it can recognize o”200K celebrities from business, politics, sports and entertainment.” as well as “9000 natural and man-made landmarks”.
​​
​​You can try out the bot yourself by adding the #vision_api hashtag to any tweet with an image. Now go and have fun seeing if the bot loves Shrek as much as Ed does.


I simply adore donkey #Shrek pic.twitter.com/F1sawshXeQ

— Sue Atkins (@SueAtkins) March 31, 2018

Top comments (2)

Collapse
 
peter profile image
Peter Kim Frank

I love these Twitter bots as a familiar form factor to interact with new technologies like this. It's one thing to host a webpage where people can upload an image and receive a result, but it's so much more native and fun (and shareable) to do so through a Twitter exchange like this.

Thanks for sharing. I'm going to have to give this a try.

On a related note — @nirzaq showed the process of building a Twitter bot here:

Collapse
 
scottcate profile image
Scott Cate

Thank you - I hope you have fun with it. have a look at the source code if you’re interested.