Conoscete il progetto narrativo Storie in XR e social VR ? Nato lo scorso agosto con il fine di far conoscere i professionisti che lavorano e vivono nel Metaverso attraverso le loro storie parla di esperienze in XR e mondi immersivi, di tecnologia e di sperimentazioni. Oggi incontriamo due artisti in un’intervista duplice, Stephen Black e Sayuri Okayama. Ho seguito uno speech di Stephen Black in occasione di un panel organizzato da Katherina Ufnarovskaia per da VR/AR Association dal titolo: “Bubiko foodtour: at the intersection of art, AR and food”. Da allora siamo entrati in contatto e mi sono appassionata alla loro storia che parla di tecnologia, food e tanta arte.
Chi è Stephen Black / Who is Stephen Black
Stephen Black è un artista e produttore in realtà aumentata, scrittore e regista in spatial film. – Stephen Black is an AR Artist/producer, writer and OARC’s 2022 spatial filmmaker.
La duplice intervista è in lingua inglese. Enjoy!
When have you approached AR? Do you remember your feelings? Please tell us your first experiences.
SB: The first time I saw AR being used was at the National University of Singapore in about 2003. What I saw was very simple; when a tablet was held over a couple of colored blocks, some text appeared. It was explained to me that AR is something like a QR code, but with more possibilities.
SO: I had heard about Pokemon Go, but never tried it. The first experience with Bubiko and AR took place in Chiang Rai, Thailand where we used a simple AR app byHewlett Packard and were assisted by a boy about 10 years old who was really clever.In Hong Kong and Shenzhen I saw other people’s projects, many of which were demos. But I would say that the 19 Crimes wine label and app was the first AR experience that really brought everything into focus as it was simple, impactful and easy to use.
In 2012, Stephen worked with Eugene Soh on a project called SPOKEN, which was a virtual gallery. They showed the works of many artists and it seemed very interesting. But it was short-lived and this made me feel a bit strange. There was so much work involved and so many interesting artists but so little public response. And there wasn’t any discussion at all about money. Now I see that it was ahead of its time, and how it connects with what is being called the metaverse. Now I understand about how AR and VR are examples of spatial computing, but I am not very good with technical topics.
Is the XR useful for an artist to express his art? Do you prefer AR to VR?
SB: Although I have a long career as an artist, and have been focused on AR since 2015, it was only last year that I could finally begin to work with GeoPose AR in a purely artistic and conceptual way.
One project was simple: A Flock of Bagels, in which bagels swirl from the ground upwards, accompanied by a soundtrack by Antonello Arciuli.
Another project, Snake Lighter, was a collaboration with Daniel Bainbridge, a New York-based sculptor. We experimented with scanning his sculptures and eventually settled upon a piece called Monkey Mop Boy, which was something like a mask. Once I had the 3D model, I used it to form patterns which then became part of a site-specific installation. I must mention the technical support we received from XR Masters, Augmented.City and Scandy. This leads me to information that answers your question: there are many technical issues to be resolved when making art with XR.
Is XR useful for artistic expression? Yes, most definitely.
However the technical and financial aspects can be significant. VR is even more challenging. For me, the biggest challenge for both is that 3D models are required. I cannot yet make a simple 3D model, let alone a complex one. 3D model makers are artists, so my projects are always collaborative. And if the models are to move, there are artists who specialize in making motions. Again a collaboration! I should mention that with Bubiko we are using motion capture to make Bubiko’s motions more natural, and this is being done with the help of Mad Marker, building upon the work of Novaby who really helped us with the initial 3D realization of Bubiko.
I read that you have been researching and testing spatial cinema ideas since 2016. How did you approach this technology?
SB: I must mention that, from 2002-5 I was the creative director for an educational 3D game making startup in Singapore. So there I learned about spatial computing, which is the “parent” of VR, XR and AR. Unfortunately the CEO of that company, and my friend, passed away suddenly- on the day of our launch! Many possibilities vanished with his passing.
To make a long story short, when the Oculus was released in 2015, I took that as a signal to enter into the world of spatial computing.
But I am not a programmer. I began to act upon ideas for a VR startup that I had been developing. But when I looked at the production costs and numbers of users for VR, and then compared them to what was being planned for AR, it felt natural for me to ficus on AR. Everyone mobile phone in the world can display AR, and because we use reality itself as a background, it can be much less expensive to create dynamic AR experiences.
Initially, the only choice was to read, study, network and learn as much as possible about all kinds of AR. I knew that I wanted to tell stories using my background as a filmmaker, but was not sure how this would be possible. I also thought there would be interesting AR possibilities for dance and performance art.
Let’s talk about Bubiko’s project. Bubiko was a co-creation between you and Sayuri Okayama. When and where did you meet and start working together? Can you explain your project?
SB: Sayuri and I met in 2011, but we decided to become involved with AR in 2016. Although we were living in Bali, we suddenly encountered visa complications, and made the decision to become digital nomads, researching both the food of Southeast Asia and the possibilities of AR. We were in Kuala Lumpur when we made the first paper version of Bubiko and took her to a food fair. We photographed her as a way to visualize what it would be like when she was a proper 3D model being seen in AR. Eventually, our networking and collected knowledge led to a presentation at Sasin University in Bangkok. Some time after that I gave presentation and workshops at Hong Kong PolyU and in Shenzhen. In January 2019, the Bubiko project was invited to MIT for the first time.
Now, thanks to the support of many people, Bubiko is an active pioneer in the world of GeoPose-based AR. She starred in historic short film made with GeoPose AR. She is still very “underground” but she is moving towards becoming a start of spatial cinema, with a strong connection towards healthy food oaf all kinds, especially those of Southeast Asia. And yes, she is experimenting with an NFT.
SO: I decided to work on AR with Stephen because we were in a very unusual situation with a lot of uncertainty. AR seemed like it would be a constant, a goal. I thought that AR would be interesting when we thought of the Bubiko idea.
Stephen had been speaking with a VC about a VR project and there was funding that was being seriously discussed. But then, unexpectedly, we had a visa issue to solve and when that was finally settled, Stephen started showing me information about AR.
It seemed like AR would become very popular as all that was needed was a phone. We decided that AR would be a better choice, because it seemed like it would be easier to create content than VR, and for a bigger audience. This was before ARKIT and ARCcore were released. When these two platforms were released we felt we made the right decision.
My background is in hotels and travel so we decide to work with these ideas and Stephen combined his art background and experiences with Cartoon Network, and we just lept into the AR world with Bubiko Foodtour.
We were very clear about an idea we called “tabletop cinema”. We envisioned Bubiko appearing on a table, combining education and entertainment. We weren’t exactly sure how this would be possible, but we just kept researching food and AR. You can see some of our progress on Bubiko’s Instagram account. At first we used paper dolls, then thanks to Novaby and Flamingo Filters, we were able to start putting Bubiko on tables and elsewhere.
I should say that Stephen and I have been apart because of Covid, which makes things very challenging as I cannot leave Japan. But still we were able to create Bubiko: First Flight, a short film in which Bubiko flies through Bari, Italy and a nice little cafe. The film has been called as historically signifiant as the world’s first website.
We just take one day at a time. We have gone far with no funding. We are looking at NFTs. DAOs too. Bubiko is ideally positioned for the metaverse: AR and VR.
What are 3 words you would use to describe Sayuri? Tell us why you choose them. And what’s about yourself?
SB: I would say Sayuri is detail oriented, cautious and creative.
Detail oriented: because she excels at planning our trips and projects.
Cautious: because she is inherently shy as well as extremely concerned about preventing mistakes from happening. She does not automatically say “no” to new ideas, but will ask a lot of questions before she feels comfortable acting upon them.
Creative: Sayuri’s creativity and sensibility have made Bubiko’s appearance and mannerisms believable and charming. As we begin to work more with photos, videos and edutainment projects her creativity will further be on display.
For myself, I would say that I am lucky, hardworking and unexcitable.
Lucky because my backgrounds as an artist, photographer, filmmaker, producer and writer are now interacting and being fully utilized as I explore the new worlds of AR and VR. I am also especially lucky toe surrounded by so many positive and supportive individuals, organizations and companies. Sayuri and I are not with a big company, nor associated with a university nor receiving grant money; we are self-funded in a world where that is extremely unusual.
Hardworking: Sixteen hours a day, or more.
Unexcitable. I cannot say I am truly patient. But whether things are going great or terrible, I think I still plod on. When things go well, hopefully that is the result of the hard work and planning. When something bad happens, I know that that is a part of life and should not be cause for prolonged unhappiness.
In 2020 you created in Bari a spatial cinema experience called Bubiko: First Flight. “ and you worked with Augmented.City. How did this collaboration with Katherina Ufnarovskaia start?
I was invited to speak at the 2020 VIEW Conference in Turin, which is a remarkable event devoted to animation and motion graphics. In order to really showcase the potential of GeoPose AR, I wanted to make an AR installation in the venue, the OGR, which is a renovated train station. The idea was to have Bubiko flying around signs, videos, balloons and festive, VIEW-related 3D models.
However to do this, I needed technical support and more. I asked Jan-Erik Vijne, the Managing Director of the Open Augmented Reality Cloud, if he had any suggestions. He suggested Augmented.City, who are doing pioneering AR work in Bari, Italy. I first spoke with Vladimir Ufnarovskii, Katherina’s partner and soon we had a plan to prepare for the OGR.
However, Covid caused a few complications, and the physical VIEW was cancelled. But I was already in Italy, and so the decision was made to create something with the brand new AR testbed that AC made. Suddenly I was learning how their AR system worked, storyboarding, working with Novaby to create a blimp for Bubiko, looking at locations, and so much more. In less than two weeks we had the basics of a short film made with AR, a film which has been said to be as historically important as the world’s first website. Have a look at this video.
You wrote the “Bubiko Foodtour’s Unusual Guide to Augmented Reality” as an introduction to AR technology. Why did you decide to write a guide?
I could talk at length about my writing and publishing experiences. But to keep it short, I wanted to create a publication that would allow me to connect to AR practitioners. I made it very clear that the book would be outdated almost immediately, and that anyone who bought it should contact me for the latest version, the most recent update.
Because I did not do marketing, I did not expect any sales. And this has proved to be correct! This does not make me unhappy! It means that now, if anyone does buy that book and writes to me, I will be forced to update it. And this will make me happy, because it will mean that I can create a version which is based on both on my recent real life experiences, as well as the latest developments in all of AR, not just GeoPose.
Next October VIEW CONFERENCE 2022 will return in Turin. If you were there which project would you present?
If I would be invited, I would share the results of the following projects: Snake Lighter, Flock of Bagels, Soda Bread in Time and Space, and, the latest AR adventures of Bubiko.
It would be wonderful to install these in the OGR.
