.png)
28.8.2023 - 11.12.2023 (Week 1 - Week 16)
Loh Qiao Yin / 0348923
Minor Project/ Bachelor of Design in Creative Media / Taylor's
University
Minor Project
[Task 1: Proposal]
[Task 2: Execution]
[Task 3: Final Presentation]
INSTRUCTIONS
/>
TASK
TASK 1: PROPOSAL
i) Project selection
We had a list of client projects to choose from, and the funny thing is most of us were in a more graphic design-ish group due to following our other friends who we had a good experience working with. After that, Mr Mike advised us to re-think our choices and choose something that would really benefit our own specialisation skills. We took a leap of faith to separate from our familiar comrades and formed a new group which consisted of just Entertainment Design and Digital Animation peeps to work on Avatar Creations.
ii) Contextual Research
We are one week behind due to a shift in groupings but without further ado, we listed out the research topics that will allow us to understand virtual avatars as much as the client does. Most of us barely knew a thing about virtual characters before this project so the journey seemed a bit daunting yet exciting. The wide range of topics was then delegated among each of us. I am more of a technical and strategist person, thus, I chose to do research on Metahuman, Metaverse avatars and influencer content creation.
Fig 1.2 Miro notes, 11 December 2023
Fig 1.3 Miro notes, 11 December 2023
TASK 3: FINAL PRESENTATION
Our final presentation went smoothly and ended on a happy note as Mr Fariz expressed his satisfaction in seeing the results. The goal of re-launching Viya with a new appealing personality and appearance has been successfully achieved!
TASK 1: PROPOSAL
i) Project selection
We had a list of client projects to choose from, and the funny thing is most of us were in a more graphic design-ish group due to following our other friends who we had a good experience working with. After that, Mr Mike advised us to re-think our choices and choose something that would really benefit our own specialisation skills. We took a leap of faith to separate from our familiar comrades and formed a new group which consisted of just Entertainment Design and Digital Animation peeps to work on Avatar Creations.
Fig 1.1 Avatar Creations project brief, 11 December 2023
We are one week behind due to a shift in groupings but without further ado, we listed out the research topics that will allow us to understand virtual avatars as much as the client does. Most of us barely knew a thing about virtual characters before this project so the journey seemed a bit daunting yet exciting. The wide range of topics was then delegated among each of us. I am more of a technical and strategist person, thus, I chose to do research on Metahuman, Metaverse avatars and influencer content creation.
Fig 1.2 Miro notes, 11 December 2023
Fig 1.3 Miro notes, 11 December 2023
iii) Software Exploration
We clarified a few simple questions with our client, especially on the software and workflow used in creating their virtual influencer - Viya's Instagram posts. Based on our contextual research, we learned that the Live Link app can be used to do facial capture to create facial expressions on virtual avatars. It is connected to Unreal Engine, a complex software that none of us knew how to use. However, we took up the challenge and began exploring the software so we could start preparing ourselves for the execution part.
We picked a ready-made Metahuman as a test subject on Quixel Bridge, which is another plugin in Unreal Engine required for creating virtual avatars. Connecting Live Link to Unreal Engine was a difficult process that we couldn't figure out as a group even after hours. Class time was over and everyone had to go home, but since I still could not let go of the confusion as to why the software was not working, I asked permission from Jamie to kidnap her high-spec laptop home to continue troubleshooting. In exchange, I lent mine to her as she needed to work on her other assignments. Long story short, I figured out the solution (Turning off public firewall protection) and managed to connect it at home.
iv) Interview
During our consultation check, we got to know we were proceeding too far ahead by diving into the software. He pulled our attention back to the right track and we continued with brainstorming backstories for Viya that could make her special in her own way. We also did research on personalities she should have based on a few successful Malaysian influencers. Lastly, we conducted interviews with 9 virtual influencer enthusiasts to gather opinions on Viya's appearance and to collect information on why people find virtual influencers appealing, which we could utilise to improve Viya's identity.
v) Proposal Slides
The master plan for launching Viya 2.0. Based on the findings and interview responses, we now have a strong reasoning behind the improvements we hope to make on Viya in terms of outfit, hairstyle, personality and type of content. We managed to get word of approval from Mr Fariz about the new art direction we planned for Viya's appearance and personality.
TASK 2: EXECUTION
i) Content Planning
The pace got quicker and quicker as we proceeded to the execution stage. While strengthening Viya's backstory, we made a 2-week content plan for Viya's Instagram posts and stories that would showcase her 3 main traits: energetic, outgoing, and authentic.
As a group, we actively searched for events Viya could make an appearance to show her relatability and relevance to real happenings around us, which could solidify her persona's believability. We had to be really strategic with the events and locations chosen as we could only spend one day on photoshooting with everyone present. The content that we could realistically execute in such a short period is just one week of plans, so we wanted to maximise the timeframe. We decided on going to Sunway Velocity Mall for the Pingmin market, a local bazaar and then to IKEA Cheras to get Viya to try out their new yogurt menu.
ii) Photoshooting
It is a blessing to have a group of jack of all trades. Our photogenic model group member, Hoi Yee, naturally struck poses with guidance from Jiahui, the pose researcher, Jamie and Jiaxien, the photographers. Seeing their enthusiasm gave me a sense of relief as I wouldn't be of help at posing or even taking photos of humans. Oh, and as a certified mom friend, I was in charge of taking care of everyone's belongings and making sure we did not cause too much of a big scene.
ii) Compositing
We then got our hands on Viya's original head file from our client, EDT. This unlocked a wide range of possibilities for posing Viya's head as we had too much limitations previously when we could only take existing photos of Viya on her Instagram to do mock-ups.
After importing it into Unreal Engine, we realised her
facial structure changed a lot compared to the
quality in Metahuman Creator. As we were running out of time to troubleshoot, we decided to send Yon, EDT's technical puppeteer an urgent help message. We were beyond grateful as he gave us a quicky reply and managed to offer guidance that solved our issue.
Fig 2.7 Troubleshoot step 1, 22 December 2023
Fig 2.8, Troubleshoot step 2, 22 December 2023
Fig 2.9, Troubleshoot step 3, 22 December 2023
Initially, we used a face rig and a library of facial expressions presets but quickly realised it is too time-consuming and did not quite nail the feeling we hoped to portray. The reason for trying out face rig was that we ran into a problem connecting Live Link app to Unreal Engine on campus. It was late at night, and we were very drained and beyond frustrated as some of us (just me) had to miss my family dinner to stay back on campus.
.png)
After a long
battle, we called it a day and went home. On the next day, Jia Xien, our Live Link saviour managed to connect Live Link to
Unreal Engine using her iPad instead of iPhone while using the internet connection at her home. We then tried the same device on campus and immediately got to work. I was controlling Unreal Engine while Jia Xien was in charge of posing the facial expressions with Live Link. I had to place cameras at a proper angle as well as lighting to mimic the brightness of
the real surroundings in the photos we took.
.png)
.jpeg)
Final Outcome
Our goal was to create a one-week content that establishes Viya's personality and appearance. There are 7 days of Instagram stories which were handled by Hoi Yee with her innate influencer vibe and making sure interactions with the followers are involved. There are 2 days of Instagram posts to showcase Viya's body language and identity. Again, the direction of her tone should always relate to her 3 main traits: energetic, outgoing and authentic.
We clarified a few simple questions with our client, especially on the software and workflow used in creating their virtual influencer - Viya's Instagram posts. Based on our contextual research, we learned that the Live Link app can be used to do facial capture to create facial expressions on virtual avatars. It is connected to Unreal Engine, a complex software that none of us knew how to use. However, we took up the challenge and began exploring the software so we could start preparing ourselves for the execution part.
Fig 1.4 Viya's official Instagram page, 11 December 2023
We picked a ready-made Metahuman as a test subject on Quixel Bridge, which is another plugin in Unreal Engine required for creating virtual avatars. Connecting Live Link to Unreal Engine was a difficult process that we couldn't figure out as a group even after hours. Class time was over and everyone had to go home, but since I still could not let go of the confusion as to why the software was not working, I asked permission from Jamie to kidnap her high-spec laptop home to continue troubleshooting. In exchange, I lent mine to her as she needed to work on her other assignments. Long story short, I figured out the solution (Turning off public firewall protection) and managed to connect it at home.
Fig 1.5 Test subject in Quixel Bridge, 11 December 2023
Fig 1.6 Imported Metahuman in Unreal Engine, 11 December 2023
Fig 1.7 Live Link in action, 11 December 2023
iv) Interview
During our consultation check, we got to know we were proceeding too far ahead by diving into the software. He pulled our attention back to the right track and we continued with brainstorming backstories for Viya that could make her special in her own way. We also did research on personalities she should have based on a few successful Malaysian influencers. Lastly, we conducted interviews with 9 virtual influencer enthusiasts to gather opinions on Viya's appearance and to collect information on why people find virtual influencers appealing, which we could utilise to improve Viya's identity.
/>
Fig 1.8 Summary of key findings from interview responses, 11 December 2023
The master plan for launching Viya 2.0. Based on the findings and interview responses, we now have a strong reasoning behind the improvements we hope to make on Viya in terms of outfit, hairstyle, personality and type of content. We managed to get word of approval from Mr Fariz about the new art direction we planned for Viya's appearance and personality.
/>
Fig 1.9 Proposal slides, 11 December 2023
i) Content Planning
The pace got quicker and quicker as we proceeded to the execution stage. While strengthening Viya's backstory, we made a 2-week content plan for Viya's Instagram posts and stories that would showcase her 3 main traits: energetic, outgoing, and authentic.
Fig 2.1 Content plan, 11 December 2023
Fig 2.2 Pingmin Market, 11 December 2023
Fig 2.3 IKEA Cheras new yogurt menu, 11 December 2023
It is a blessing to have a group of jack of all trades. Our photogenic model group member, Hoi Yee, naturally struck poses with guidance from Jiahui, the pose researcher, Jamie and Jiaxien, the photographers. Seeing their enthusiasm gave me a sense of relief as I wouldn't be of help at posing or even taking photos of humans. Oh, and as a certified mom friend, I was in charge of taking care of everyone's belongings and making sure we did not cause too much of a big scene.
Fig 2.4 You go girls, 11 December 2023
We then got our hands on Viya's original head file from our client, EDT. This unlocked a wide range of possibilities for posing Viya's head as we had too much limitations previously when we could only take existing photos of Viya on her Instagram to do mock-ups.
Fig 2.5 Viya's head model file (Metahuman Creator), 11 December 2023
Fig 2.6 (Before vs After) Quality difference of same head model, 11 December 2023
Fig 2.7 Troubleshoot step 1, 22 December 2023
Initially, we used a face rig and a library of facial expressions presets but quickly realised it is too time-consuming and did not quite nail the feeling we hoped to portray. The reason for trying out face rig was that we ran into a problem connecting Live Link app to Unreal Engine on campus. It was late at night, and we were very drained and beyond frustrated as some of us (just me) had to miss my family dinner to stay back on campus.
.png)
Fig 2.10, Face rig and facial expression presets, 11 December 2023
.png)
Fig 2.11 Unreal Engine in action, 11 December 2023
.jpeg)
Fig 2.12 Unreal Engine in action, 11 December 2023
Live Link
We experienced lots of trials and errors and here are some future notes for myself
1) Secure a strong network connection
2) Strength of device detecting the network connection matters
3) iPad worked too when iPhone did not
4) Disable the public network firewall
iii) Posing Facial Expressions
Jia Xien was in front of her iPad making different varieties of facial expressions through Live Link and kudos to her for having to do it in public. I was behind the laptop controlling lighting, angles and capturing the expressions in Unreal Engine (and hyping her up). I screenshot the facials instead of rendering it as the laptop was very laggy + running out of time to figure out how to render through the camera view.
Compromise: Sharpen the facial images during post-editing, since it will be posted on Instagram where the quality of pixels is low, it would not be obvious
There were a total of 12 chosen images for Instagram posts, thus there were 12 different facial expressions and head angles we had to mimic from the plate image of the model including the lighting direction. Below is just a small chunk of facial expression screenshots, we made sure to take as many as we can.
iv) Editing
There were 2 stages-> First was connecting the head with body and basic colour correction, Second was applying ambience lighting and minor fixes on anatomy issues. It is necessary to split it into two rounds and be done by different people as the eyes start to adapt and get used to any minor flaws when focused on a picture for too long. Jamie and Jiahui were working on the first round of editing while I worked on making sure the overall image was cohesive in the 2nd part of editing.
We experienced lots of trials and errors and here are some future notes for myself
1) Secure a strong network connection
2) Strength of device detecting the network connection matters
3) iPad worked too when iPhone did not
4) Disable the public network firewall
iii) Posing Facial Expressions
Jia Xien was in front of her iPad making different varieties of facial expressions through Live Link and kudos to her for having to do it in public. I was behind the laptop controlling lighting, angles and capturing the expressions in Unreal Engine (and hyping her up). I screenshot the facials instead of rendering it as the laptop was very laggy + running out of time to figure out how to render through the camera view.
Compromise: Sharpen the facial images during post-editing, since it will be posted on Instagram where the quality of pixels is low, it would not be obvious
Fig 2.13 Live Link in action (pardon the poor video
quality), 11 December 2023
There were a total of 12 chosen images for Instagram posts, thus there were 12 different facial expressions and head angles we had to mimic from the plate image of the model including the lighting direction. Below is just a small chunk of facial expression screenshots, we made sure to take as many as we can.
Fig 3.1 Screenshotted facials, 11 December 2023
Fig 3.2 Screenshotted facials, 11 December 2023
There were 2 stages-> First was connecting the head with body and basic colour correction, Second was applying ambience lighting and minor fixes on anatomy issues. It is necessary to split it into two rounds and be done by different people as the eyes start to adapt and get used to any minor flaws when focused on a picture for too long. Jamie and Jiahui were working on the first round of editing while I worked on making sure the overall image was cohesive in the 2nd part of editing.
Fig 3.3 (Before vs after) Photoshopped images, 11 December 2023
Fig 3.4 Lighting and colour correction, 11 December 2023
Our goal was to create a one-week content that establishes Viya's personality and appearance. There are 7 days of Instagram stories which were handled by Hoi Yee with her innate influencer vibe and making sure interactions with the followers are involved. There are 2 days of Instagram posts to showcase Viya's body language and identity. Again, the direction of her tone should always relate to her 3 main traits: energetic, outgoing and authentic.
Fig 4.1 Pingmin market outing post, 11 December 2023
Fig 4.2 Pingmin market outing carousel post, 11 December 2023
Fig 4.3 IKEA outing post, 11 December 2023
Fig 4.4 IKEA outing carousel post, 11 December 2023
/>
Fig 4.5 Compilation of execution, 11 December 2023
Our final presentation went smoothly and ended on a happy note as Mr Fariz expressed his satisfaction in seeing the results. The goal of re-launching Viya with a new appealing personality and appearance has been successfully achieved!
/>
Fig 4.6 Final presentation slides, 11 December 2023
FEEDBACK
Recorded in Project Document
https://docs.google.com/document/d/1FooSrRvhUj9ThwTQcU1eNu_-jfwj9FjsCPY2bvgNtOg/edit?usp=sharing
Recorded in Project Document
https://docs.google.com/document/d/1FooSrRvhUj9ThwTQcU1eNu_-jfwj9FjsCPY2bvgNtOg/edit?usp=sharing
REFLECTION
At first, all of us were demotivated by the workload scope that was focused on marketing and researching. However, I still appreciate the opportunity to be exposed to social media content management. During the execution stage, we were exposed to learning Unreal Engine by ourselves from scratch and it really tested our perseverance due to the ups and downs of facing technical issues. The hard work was really worth it when we saw a compilation of our Instagram stories and posts of Viya. We felt even happier when we got a word of acknowledgement from Mr Fariz saying it feels believable like a real influencer.
I feel beyond lucky and grateful for having the opportunity to take up this project with the Experiential Design Team and interact with Mr Fariz Hanapiah, who is well-known from my view in the VR/ AR industry in Malaysia. I felt a bit star-struck when I found out he is involved in some of the amazing digital art installations I've seen at events. Lastly, none of this would be possible without my fellow group members, whose personalities sometimes scare me, amuse me, or make me question my existence, but mostly have the best teamwork spirit which got our team through all the hurdles we met throughout the project. Viya La Vida!
I feel beyond lucky and grateful for having the opportunity to take up this project with the Experiential Design Team and interact with Mr Fariz Hanapiah, who is well-known from my view in the VR/ AR industry in Malaysia. I felt a bit star-struck when I found out he is involved in some of the amazing digital art installations I've seen at events. Lastly, none of this would be possible without my fellow group members, whose personalities sometimes scare me, amuse me, or make me question my existence, but mostly have the best teamwork spirit which got our team through all the hurdles we met throughout the project. Viya La Vida!
Comments
Post a Comment