This is a concept Cortana appliance I've built. It's basically what I imagined Microsoft's version of Alexa or Google Home would be like if they were to use the holographic AI sidekick from the Halo franchise. While still a work in progress, I just wanted to show how it's coming along.
So, this a Pepper's Ghost hologram that uses Windows 10's native Cortana experience. It's all powered by a Windows 10 device with 4GB of RAM and a built-in Arduino used for controlling the platform lights. There's a portable USB monitor on the top that reflects on 3 panes of mirror glass. The enclosure itself is a custom design I modeled. I then 3D printed each part of the assembly in PLA. There's also an omnidirectional microphone that sits on top of the unit, and a decent mini speaker built into Cortana's pedestal on the base.
The software it runs is essentially in two parts: A Unity 3D app that presents and animates Cortana in three different camera angles and communicates to a second app: a proxy service that parses data going to and from the native Cortana web service. The proxy app renders the HTML from Cortana result queries which is then presented in the Unity app. It also features real-time face tracking with the front-facing camera which moves the rendered camera perspective relative to a single viewer's head position - this makes it look a bit more 3 dimensional when facing it. The app also controls an LED strip surrounding the edge of the platform.
Cortana's animations were recorded with the help of my lovely wife who regrettably endured many takes of motion capture in our living room. I used dual Kinects to record her motions and applied them to the Cortana model in Unity. Other animations would include an array of screens that Cortana brings forth with her hand motions. Each screen displays a unique mock UI loop and each loop plays on a random screen at start. This makes the screen animation semi unique for each query. I have more animation ideas for query processing, but have not gotten around to it.
The query's answer card is rendered and presented on one of the screens and is flipped around to the viewer. This was done by first rendering the html in a headless browser, then modifying its content to look good on a dark background.
Knowing when the native Cortana was being displayed and hidden was quite the challenge. Because Cortana is a UWP app, its sandbox design prevents Window Handle api calls like Win32 apps, locking me out from reading its UI content. I ultimately settled on monitoring a single pixel color in the taskbar to determine when Cortana's UI was present.
As you can imagine, since this is all using the native Windows 10 Cortana, this device can be used not only for knowledge based queries, but also for home automation and music tasks too. I can extend functionality by simply building UWP apps that can fulfill Cortana integration and use them accordingly. I can perhaps demonstrate this in the future, but that's it for now
Here's a gallery containing additional build photos:
Cortana Hologram Concept
For direct inquiries regarding this project, reach me at cortanahologram@unt1tled.com
Press and Feedback
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
|
|
|||
|
|
|||
|
90 Comments
Jan D
Wow, this is really awesome! Great work!!!
Austin
This is amazing, are you planning on selling them?
Jonathan
You Sir, are the real deal!
Tim McDowell
I would love to build one of these or play with the code to create a Google home version. Any chance this project source will be released?
Joe
Wow...amazing. How is the laptop screen projected down into the pyramid?
S W
It's just sitting on top of the pyramid. You can find plans on youtube to make those little pyramids for your phone with clear plastic and tape! Look for diy iphone hologram on youtube. then you can play specially formatted videos to generate the illusion.
David Emmanuel Jarvis
Great Work !!! I am very impressed !!! Keep up the great work !!! The Future is waiting on you !!!!
David Emmanuel Jarvis
I sent your YouTube Video to Bill Gates !!! Lets see.....
GaTechThomas
Mad skills. Wow.
James Burton
I want this so bad!!! Congrats on making this mate, I'm sure you'll be creating many more items here soon now! Good luck
Dee
please open source your programs!
Michael
You did a great job. Any chance you will publish the STL's for the 3-D printed parts. That's a beautiful ghost projector, and I'd love to build one even if just to learning more about making 3-D projection files.
Also, that bottom ring with the LEDs, would make an awesome gate for a mini-drone.
Thanks.
M
Luca
Are you planning on selling these? I would totally buy one
_ThaNerd_
Dude this is awesome!! Great work!!
Markus Dei
Fascinating and truly amazing. The future of telecom and digital media! My friend you will be very rich after this concept breaks ice with the mainstream computing industry. Well done!
Duylam
Broooooo I need this for my Xbox asap
Jamie
Hey man, I just saw this posted on Reddit. I'm a software developer as well, and have a school that teaches this kind of awesome tech for kids. Would love to chat with you about bringing this stuff into our curriculum!
Johnny Baillargeaux
Hi,
Very interesting !
Do you have any details regarding the cortan web service ?
I know that microsoft provides an sdk but by looking the printscreen, I have the feeling that we can query cortana via a web request.
Thanks
Oliver
Great job! keep it on! http://ncomputers.org/news
Aceconklin
Shut up and take my money!
Matt Freestone
Please share the specs of everything you bought, the .STL files (I have a 3D printer) and the compiled code, etc :-) I want to build one too. You did AMAZING work here man!
Matt Freestone
Also, I have a massive amount of home automation deployed and use Alexa (Echo devices) to voice control everything. Prior to Cortana you could voice command with INSTEON but last I heard Cortana doesn't have any home automation integrations. Has that changed and I missed something?
John
Dude you're fucking amazing! I hope Microsoft contacts you and this ends up becoming something even more amazing!
Shilonda
Hi Jarem,
If you need help on the administrative back end; we'd be honored to help with this project or any future projects. My company has been in business since 2006 and we can help you with press as well; so that you can spread the word even further regarding your incredible design. Looking forward to chatting. Schedule some time at your convenience: https://virtualworkteam.setmore.com
Shilonda Downing, Founder
Virtual Work Team
https://virtualworkteam.com
Gabriella
Can you please post a DIY on Reddit?
Thank you!
Brandon H
That is totally awesome and I am adding it to my DIY bucket list.
Is it possible to put the monitor on bottom and have the images reflect onto the inside of the glass.
It would be awesome unless physics does not allow this type of hologram to work that way.
I am thinking how to get it as close to free floating as possible.
Kamal
Great work!
Anthony H
Jarem,
Fantastic work! Really impressive tech demo.
LikeMind
I was literally pitching this idea to a co-worker two days ago, and I'm crushed/impressed that you already have a working demo. May I be so bold as to recommend NOT releasing it to the open source/hobby crowd? The level of interest you are seeing just indicates how successful you could be with it. Good luck with it and hit me up if you want to work with someone on the home automation side...
Brandon L
I'll pay $500 right now for one.
JTB
Hey, this is amazing. Like "LikeMind" I have had this idea stewing for WAYYYYY longer. Like, I've drawn up designs and everything. My issue is I have NO experience in programming besides taking online self-teaching classes.
How did you originally design this? Have you sought any other sort of applications besides home automation/assistant? My idea was to make something that could have that capability, but also the ability to have simple 3D games (Chess, it was all about Chess).
I must say, this is so depressingly awesome, but I am very proud that there are people out there truly innovating beyond what people could ever expect. Thanks for allowing us all to share this and praise your work. It's absolutely inspiring.
Also, I have no advice for how to go from here. You will have THOUSANDS of people trying to pressure you. I wouldn't be surprised if you even received a lawsuit or two! (Despite how that may seem, it's actually a HUGE sign of success if people want to steal YOUR work through lawsuits. It means you did what they can't and only dream of). I wish you success, strength and thank you for your amazing work.
Over and out. Also, please don't share my email address, as that is my name.... lol
Bruh
Could you post source code?
Fernando Torres
It's very amazing, I've very impressive the movement of the hologram and its details like the movement of the windows
Congratulations friend
Jus
I want you to get noticed by Microsoft. I'm so motivated right now... I'm social mediaing the heck of this! Great work!!!
Grant
Wow, great job!!! I can't wait to see more ;)
Franco Sechi
Wow this should be in stores or available for sale online!! With mass production the cost should be low!!
Xeno
OMG! Shut up and take my money!
Please....bring it to kickstarter!
Jesse W
Is there stocks/shares for sale?
Kevin Jennings
Awesome work!
Kevin
MUST HAVE! Please do a kickstarter. I need one of these! I can't imagine the amount of time this took but this is straight up Awesome. Well done!
Walt
This is exceptional, and like many others, I would like to be able to create like holograms when and if the prototype is available.
Shane
This is great stuff. Particularly interested in the fact that you used Unity as the 3d engine. We code in Unity.
Pierrestéguy
Bravo ! C'est génial !
Solomon Rothman
This is absolutely amazing! The fact you were able to put all these pieces together into a working concept yourself is incredible! I hope you get offered a high paid job from all this exposure. I'm sure you will be. If you like the project enough to continue and expand it, I bet this could be funded on kickstarter within 48 hours and you could really develop it more.
Elizabeth Rivera
Brilliant work! Are you open to a conversation with us?
unt1tled
Sure! Reach out to me on Twitter, Elizabeth or I can email you.
K. Mosby
Fantastic work! How can we contact you directly?
Rashad
This is very awesome, I would like to purchase one or even invite you to my kids robotics and science classes. The is something the youth would love. Good job man, and good job to your wife for helping you.
Mako
I would so buy that! Not really impressed with the Cortana assistant, but that application of it is killer.
Christian
This is awesome! Would it be possible to get the 3D models for the enclosure to print one out myself?
Payden
This is freaking awesome!!!!!! You are the real MVP!!!!!
Adam
While the Halo 4 Cortana model is absolutely the best... since this is the first version, you probably should have used the Halo CE model. :)
Ross
Hello,
I'm not sure if you know this but eBaum's World posted your video. I am extremely intrigued by your work, but unfortunately don't have the advanced knowledge or creativity you possess. If you decide that you would like to sell one fully completed, or even just the frame I would be extremely intrested. It is 1:33 a.m. where I live right now and after seeing your video, I knew I couldn't just skim over it. I wish you the best of luck with this venture.
Best,
RW
Klaus Kühne
Bring it to kickstarter!
Greg Whiteside
Hi JA
My name is Greg Whiteside
We are working on the 4th season of a major network television show and would like to discuss using your amazing device as a prop piece in a set for one of our episodes.
Please contact me.
Cheers
Greg
Christopher
Absolutely great work, name your price and I will purchase one! Well Sir!!!
Jerry Hajek
Outstanding, Greg! DIY efforts like yours show that 'garage tinkerers' can still show the 'big guys' a trick or two on a 'relative shoestring' is possible. Personally I'm working on a Walsh omnidirectional speaker for my own amusement with some decent results...
https://www.youtube.com/watch?v=10HjqHXpZtU&feature=youtu.be
Crude, but the next version (underway) ought to be an improvement both sonically and visually.
Carry on!
David Djoho
Bro this is amazing. i would love to support your idea to take this even further. im a team player, im not looking to get payed. but if its not possible, i stilll support this project 100 percent.
FUNN Magazine
WOW! Great work! Can't wait till these are available!! Hurry up and take my money!!! Gimme gimme. also... We'd like to do an article on you for our holographic magazine. Reach out if you like our work. funnmag.com/2017promo Cheers!
carlos isasmendi
Excellent, I'm trying to make the 3D model still, it's hard to cheer. I'm going to try out a projector. I'm from Argentina and here we do it well home
Calme Ishmael
Duuuude! That is the COOLEST thing I've seen in a long time! This would be the perfect thing for any super nerds home. I'm dead serious when I say, I'll happily pay you for one of those! I'm a deploying contractor with no time to make 1 of my own. Please private message me, I'd love to give you some money for either a completed one, or a full preprogrammed parts kit.
Eric
Kickstarter! Here take my money!
Josh Arcadia
Keep working! so one day mark Zuckerberg get jealous and give tons of money
Rossana
We are working in the same direction, I am also using real time body detection in order to get things. I arrived to your site while building mine (I am in a beginner phase regarding this aspect). Good work!!, very inspiring.
cybernesto
You sir are a genius! Love this project. Keep up the amazing work.
Rigo
Man, you are a master. I hope Microsoft watch your amazing work!
Remy Sheppard
Dude! This is so sick. Congrats on the awesome work and the subsequent recognition for it!
Raymund Paglinawan
Nice! I hope this could make into market. Great job!!
Ed
Launch a Kickstarter!
Austin
Please, take my money. I want this in my life.
Stephan Gerard
Dear Mr. Archer:
Like WOW!!! You are the cutting edge of interpersonal computing technology. Its like you are someone from the future come back in time to show us primitives what its like in the future.
You are a genius in this field of tech!
Can I please ask you for detailed parts and equipment to be able to build one of these myself, and
as a respectful person I am put your name on it as the inventor?
If I were Bill Gates I would be offering you a job at Microsoft paying you $2,500/hr to think up stuff that would make Microsoft a way cooler and better place.
Let me know if you would honor me in letting me know the list of things to purchase to make this please. I thought up of a business idea of making things with a Makerbot 3D printer and an Avail360 metal printer to make simple house hold items and with the metal printer making things like jewelry i.e... rings off the two 3D printers and selling them on eBay.com, and Amazon.com. First though make the sale on the web sites then produce the item on the 3D printer and then ship it off to the buyer. I would not do this to your item, you need to do that. If I know my tech people you could be a billionaire within a year with this product!
Hope to hear from you soon. This blow Google home out of the water!!!
I am Respectfully:
Stephan Gerard.
Jason
how did you make the cortana AI animation?
Verdi Pratama
Wow, this is amazing! Are you planning on selling these?
Michael Crasselt
Thats so cool but if you use the Google assistant you have a better k. I.
Jorge Almeida
MAGNÍFICO!
Moti
This is so freaking cool. I love it.
NTKelly
This is outstanding work, Congratulations. I would like to know if are selling them now or in the future?
and would like to be informed if you post any more videos or information.
thank you for showing us this.
Marvis
Jarem,
My sister (Alessandria) just mentioned you to me after she asked me what I was working on out here in Houston. I mentioned a website and your name came up and she directed me to your site. Keep up the good work young man and I hope to meet you one day.
Tauheedul Ali
I'm glad my tweet about this was picked up by Neowin and MSPowerUser :)
Really pleased to see it has gathered a lot of attention, well done Jarem!
Please continue to come up with more exciting ideas!
Renato Mari
Greetings from Brazil! First of all congratulations for your initiative, that's an amazing project you are working on! I'm enrolled at an Engineering course in which I have this discipline called Optics. We are planning on making a big hologram to present it at our university's science&phisycs fair. I understand you won't share much of the details because of obvious reasons. However Id like to ask you the dimensions and thickness of the glass you used so we can reproduce this in class! It would be amazing if you did! Again, congratulations and good luck!!
Renato
Megan Ruiz
Hi, I'm thinking of doing a similar project to display my programming skills and I was wondering if I can ask you advice. I was thinking of doing a hologram person in a mirror setup that answers questions a lot like siri. If you can give me any pointers I would greatly appreciate it.
rahul
how did u connect monitor to cortana .?. how is video coordinating with cortana ?
how did weather reports come there ... sir please answer to the questions .. i am up to build a similar model . how can i contact you ?
best upright vacuum
Upright cleaner removes the dirt, dust, and pet hair out of even the deepest carpet, and include on-board specialty tools for hard-to-reach areas.
derko
TAKE MY MONEY AND SHUT UP !
I NEED THIS
Chris
Hi, I've been checking on this for a couple of years now. Is this for sale, and if so how much is it and where can I purchase it? Great job on this project, truly sensational
Kevin Butcher
this is amazing ! im trying to work out how to do something similar on the front of a home theatre case ? i also trying to work out would the effect work if the pyramid was in a acrilyc coolant tank with clear fluid and a screen below it ?
Shakil
You're gonne revolutionize the holographic concept with this.... Please work on how you can make this interactions smooth and more human... Like expressions, body movement and also walking.. Great work.... Please finish it fast and keep it in our reach so that it can be used by billions not thousands...
Beautifullll. Please keep us updated on improvements...
Jason Sutton
I would love one of these. How much do you charge?? Cortana is my favorite helper. That’s you
Jason Sutton
Inside Sales Rep.
Ferguson Waterworks
918-645-8343
parimatch.cyou
Букмекерская контора Париматч. Делайте онлайн интернет ставки на футбол, теннис, баскетбол и другие виды спорта. Ставки на спорт - БК <a href=https://parimatch.cyou/>Parimatch</a>. Так же есть ПМ казино, которое имеет лицензированные игровые автоматы!
parimatch.cyou
Международная компания <a href=https://parimatch.cyou/>бк пари матч</a> в сфере игорного бизнеса более 20 лет. Бренду доверяют игроки по всему миру. Каждый день онлайн площадка делает все, чтобы игроку было интересно. Площадка предлагает свою линию для ставок, а также предоставляет весь спектр казино-продуктов. Игрок может сыграть в виртуальное казино, классические и современные аппараты, либо игры собственной разработки.
Leave a Comment: