Vocaloid
Abstract
Our goal with this paper is to provide an understanding of what “VOCALOID” actually is, how it came into being and how its popularity evolved up until now. We will analyze the first VOCALOIDs that were available to the public, LEON & LOLA and especially take a deeper look at Hatsune Miku, the first Vocaloid developed by Crypton Future Media. By analysing different vocaloid songs on structure, composition and the specific elements it contains we have also created our own Vocaloid song. To define how Vocaloid popularity evolved throughout the last decade we collected data from Spotify & VocaDB to look for patterns in songs or the general rankings and thus will try to formulate a conclusion if it is still relevant up until today.
Song:こんにちは皆様 feat. Yuzuki Yukari(結月 ゆかり)
For this project We’ve created a song from scratch with the official Vocaloid 5 software. Before reading any further, we recommend you to listen with a good pair of ear/headphones to the song to get a first glimpse into the world of Vocaloid. A deeper analysis on the software and ‘the making of’ can be found under the chapter: ‘Song breakdown’.
The First Generation “V1”
The term VOCALOID refers to a singing synthesizer application software, which is considered to be the brainchild of Hideki Kenmochi. Developed by the YAMAHA Corporation, the project began in 2000 with the goal of providing human-like vocals to music producers, fast and at a low cost, giving them full control over the music production. The software gets complemented with voice-banks which are made up of vocals, provided by a voice provider. These voicebanks are then accompanied by an illustrated character or avatar.
In March 2002, the VOCALOID project, known under the codename “Daisy Project” entered a proto-type stage and was subsequently licensed out to various 3rd party studios. Crypton Future Media were the first to join, followed by Zero-G Limited and PowerFX. Thus, both English and Japanese voicebanks began development.
The name “Daisy” was later dropped due to copyrighting conflicts and in 2003 the name “VOCALOID” had been decided, an amalgamation of the words “Vocal” and “Android”. Among the 4 known vocals for “Daisy” (LEON, LOLA, HANAKO and TARO) only LEON and LOLA were shown to the public on January 2004 at the National Association of Music Merchants (NAMM) Show in the U.S. They were both released as official voicebanks for the final VOCALOID software and were eventually followed up by MIRIAM and MEIKO, the latter being the first Japanese VOCALOID.
It’s notable that VOCALOID was released towards the end of the “Flash golden age” (FLASH黄金時代 ; 1998-2002/2005), a period known for the rise of flash-based productions and the birth of file sharing sites, such as YouTube.
LOLA & LEON are respectively the female and male English V1 voicebank with soul singer vocals, developed and licensed by Zero-G Limited. Released in January 2004, neither of them has an official avatar, but they’re represented as stock images of a pair of lips. Although they don’t have an official avatar a lot of fans try to portray them by making fanart. Fans often depict them as a blond haired boy and a black haired girl. They are counterparts to each other. Both of them received the EM (Electronic Musician) Editor‘s Choice Award in 2005. I think this was an award presented at WIRED Nextfest 2005, but searching via DuckDuckGo and Google provides no clear reference.
They both received standard advertisement at the time, mainly in electronic magazines and on the internet. Because there was no VOCALOID fandom, their marketing was aimed at professionals and had no fan input in mind.
Their voice providers have not been revealed, but are officially described as “Black and English”. They are both well-established musicians in Great Britain. LOLA’s provider has its origins in the Caribbean, which leads to LOLA reverting to a Caribbean accent when used outside of the soul music genre. Despite this, LOLA received much better reviews than her male counterpart, but both of them failed to impact the American market due to their accents.
LOLA is featured in 38 songs and 4 albums. LEON is featured in 44 songs and 11 albums.
Zero-G permanently discontinued sales of LEON & LOLA as of September 2015.
Hatsune Miku
Hatsune Miku, a name that might sound familiar to those who peek an interest in Japanese pop-culture. When talking about Vocaloid she is one of the topics that can’t be left out. Depicted as a 16-year-old girl with long turquoise pigtails, Hatsune Miku has been able to attract thousands of fans to stadium concerts all over the world by performing her many songs. She isn’t anything like your regular artist, she’s actually a hologram that was developed by one of the leading companies of music technology: Crypton Future Media using the Yamaha software. Furthermore, she’s the second Vocaloid using the Vocaloid 2 software and the first Japanese Vocaloid to use the Japanese version of this software engine.
After Hatsune Miku was released in August 2007, there was a giant amendment within the Japanese video website Nico Nico Douga. Individuals began to post their original compositions (with video) to the site, and a few of them got quite popular. The people who saw those videos began to think that they might be able to do something similar, so they went on to purchase Hatsune Miku, and began to create their own original compositions. A lot of positive feedback took place, and Hatsune Miku still remains to sell well. It sold-out over 50,000 units, a record in the field of software synthesizers. It's not restricted to an easily-made musical creation, some people began to add different videos. Others liked the music and modified the lyrics, re-mixed the music and reposted it. Or even some who liked the music sung the song and re-posted it. A lot of personal modifications were made based on personal taste, which makes Hatsune Miku a character adjustable to anyone’s preference. And with more than 15 million views and uploaded just 20 days after the official release of Hatsune Miku, ika feat. Hatsune Miku - "Miku Miku ni Shite Ageru♪" みくみくにしてあげる♪ 【Shiteyanyo_ is the music video containing Hatsune Miku content that is the most popular on Nico Nico Douga. Only based on this example we can sense the popularity of Hatsune Miku herself.
【LEON & LOLA】 World's End Dancehall (English Ver.) 【MMD + VOCALOIDカバー】 - YouTube
Maker Unofficial: Hatsune Mix
The popularity of Vocaloid and especially Hatsune Miku reaches way further than only the music. Many manga series, videogames and other merchandising were created, marketing the product, and bringing it over to other countries.
The original artist of Vocaloid 2: Hatsune Miku also wrote and illustrated a Japanese doujinshi manga called Hatsune Miku: Unofficial Hatsune Mix, originally all based around Miku. Later, as the series progressed, other characters from Vocaloid 1 and Vocaloid 2 were introduced in the story such as Kagamine Rin and Len, and Megurine Luka to name a few. The manga was originally published by Jive’s Comic rush, a shōnen manga magazine, from January 2008 and December 2010. However, the characters and settings that Kei depict in the manga are not official, like he said in an interview. It was licensed for distribution in North America by Dark Horse Comics in the summer of 2014, so official English versions of the comic are available. Beside this series, there are tons of other manga works, not only about Miku but also about her other Vocaloid friends.
As for anime there isn’t yet an anime specifically about her. Maybe in the future they will work on that, but for now she has only disappeared in several anime as a side character.
Hatsune Miku -Project DIVA- series
Hatsune Miku -Project DIVA, also known simply as Project DIVA, and abbreviated as PD are a series of video games by SEGA starring Hatsune Miku and other popular vocaloids, even including popular fanmades. When playing the game, the player aims to press a certain sequence of buttons to keep in play, this is also known as a rhythm game which is in general a quite popular game in Japanese arcades among the youth. Because of its popularity, Project Diva became one of Sega’s longest-lasting franchises and still has games being made now. Hatsune Miku became one of SEGA’s mascots and together with the other popular Crypton vocaloids, she became an official and copyrighted SEGA character.
It was a real struggle for Vocaloid fans outside of Japan who desperately wanted to play these games, since they were only available in Japan, and therefore only in Japanese. After 2014 they started to release English versions, which made them also available for a broader public. Last year, the first game for a non-PlayStation console was released. It is a rhythm game in the project DIVA series, for the Nintendo Switch. Later there was also an English version of this game released, which made it available in North America and Europe and thus contributed to another rise in popularity among a new generation.
Concerning previous research
When looking at pure academic sources concerning Vocaloid not a whole lot of articles have been written up to this point, but seeing as Vocaloid is a rather new phenomenon this should not come as a surprise.
Most academic articles limit themselves to explaining what Vocaloid is to a wider audience but seeing as this was already explained in the introduction I will not touch upon this any further. In addition a lot of articles also talk about the most famous exponent of the Vocaloid phenomenon, Hatsune Miku, but who this character is and how she was created will be looked at separately in the course of this project due to her significance for the Vocaloid scene.
What the academic sources mention besides what has been mentioned beforehand is that recently Vocaloid is used more and more in Japanese ballads, songs, movies, bunraku performances and many more. Not only domestically but also internationally as a way to promote products of Japanese culture abroad. Therefore it can be safely stated that Vocaloid is yet another way Japan can manifest itself as a “soft” power in foreign markets.
One article looks at the diffusion of user generated content with regards to Vocaloid and draws the conclusion it is hard due to lack of mass marketing, however platforms like Niko Niko Douga are a way to get noticed.
When trying to get an idea of the popularity and reach of the Vocaloid phenomenon both inside and outside Japan several (non-academic) sources give a good indication of its popularity. One thing that is particularly interesting is that Vocaloid clearly evolved beyond an application that was initially only used in the music industry. Vocaloid has established itself as the face of young Japanese otaku as a means to distinguish themselves. Vocaloid takes over the teen karaoke scene in Japan, more than half of 2014's top 20 songs were synthetic. But it doesn’t stop there given the fact that tablets with Vocaloid driven educational software are being used in Japanese elementary schools in order to replace paper textbooks.
More recently the Japanese government has appointed Hatsune Miku as corona countermeasure spokesperson in order to use Vocaloid as a communication device between itself and its younger citizens. To strengthen the message of wearing mouth masks in public they made a video of Hatsune Miku wearing one. This shows us that she’s still a very popular character able to influence people’s thoughts.
When considering the international appeal of Vocaloid, especially Hatsune Miku has paved the way for success abroad. Not only has she sold out concert tours in America and Canada but she does this while collaborating with internationals stars like Pharrell Williams and Lady Gaga even made her an opening act during one of her concert tours. Hatsune Miku also made a guest appearance on the Late Show with David Letterman who was America’s most popular and most watched TV host at the time.
Recently a theme park dedicated entirely to Hatsune Miku was opened up in China demonstrating that there is certainly a possibility for Vocaloid to tap into the very important export market over there.
However despite this we found out that a few years ago, in 2017, someone also did research about the popularity of vocaloid. More precisely, the decline in popularity since 2012. With the main argument that there is a decline in songs that have been uploaded. She did the research looking into two communities: the English-speaking and Japanese -speaking, which are the two largest communities. The number of songs that were uploaded was investigated on NicoNico only, since youtube mostly has re-uploads of the vocaloid songs. There was also a survey done on people in the vocaloid fandom, and data of search terms for as well google as youtube were gathered. This data was gathered with google trends for 6 common search terms related to the topic as well as their Japanese equivalent. Besides that, there is a person named Fritz who is a big fan of vocaloid and runs a web magazine called ‘VOCALOID ORIGINAL’ where he posts statistics and information of original vocaloid songs. Graphs that were taken from this website were used as a secondary source.
While analysing this data, you can conclude that after the peak of popularity in 2012, there was an decrease in search interest in Vocaloid-related terms, suggesting a decrease of interest in Vocaloid. The same goes for the graphs from Fritz, where we can see that the weekly and monthly uploads of vocaloid songs decreased after 2012. However, It is difficult to draw a conclusion out of the survey, since the popularity of vocaloid is both increasing and decreasing according to the participants, with a very small difference in the percentages (Yes = 49.2% and No = 50.8%).
For our research we wanted to show the popularity of Vocaloids, but this investigation shows us that the popularity of vocaloids has started to decrease since 2012. The reason for this is that there have been less songs produced and uploaded and this leads to less interactions between fans and vocaloid. It is said that there haven’t been as many good songs and songs of famous vocaloids in comparison with previous years, which is also a reason for the decline in popularity. In addition the vocaloid culture has also changed over the years into other forms, including mainstream popularity and they put more emphasis on the voicebank Hatsune Miku rather than other voicebanks. This could also be an explanation for the decrease in popularity since there is a lot of discussion about this topic. People's opinions are strongly divided when it comes to the use of voicebanks, which one is better and which one is worse.
Some fun facts
So we can conclude that the vocaloid culture has decreased over the years, but it still plays a huge role, especially in Japan. Vocaloids are still popular for their songs and dance performances, but besides that they are also used in commercials for example hair shampoo commercials, games and many more. If that isn’t crazy enough vocaloids are also a source of gossip and a very interesting topic to talk about in tabloids. To give you an example we read an article about Akihiko Kondo, who is very much into the vocaloid artist Hatsune Miku. He’s a 35-year old teacher in Tokyo and already knew from a young age he would never marry or fall in love. But then he found his one true love Hatune Miku and decided to marry this 16-year old hologram. You can click this link here if you want to get a deeper insight in his story.
Besides that, Vocaloids are also real fashion icons. For example, Hatsune Miku has already worn outfits that were especially designed for her by Marc Jacobs and Louis Vuitton. Because of stories like these the vocaloid culture gets more attention from other audiences as well, but not always necessarily in a positive way.
Spotify & VocaDB results
Spotify scraping
To begin the scraping for Spotify, the most promising tool I first came across was the Spotify Scraper. Uses Python and the Spotify Web Player to scrape and extract data, with or without a Spotify account. First I downloaded the latest version of Python and tried to use the code included on the website to install the Spotify Scraper, but after a quick search I found I had to write the code in the terminal of the PC itself, not the Python terminal. The website included various ways to install the scraper, but after multiple errors, I found out the code only worked on Linux operating systems and not having the necessary knowledge to rewrite the code myself, I looked for another way to install it. PyCharm, an Integrated Development Environment (IDE), made it possible to directly install from inside the program without the use of code, while also doubling as a terminal. Using this program, I was able to install the scraper and the request library needed for it to function. The website also lists various additional options such as using cookie files, setting a header and setting a proxy, but these are not necessary to use the scraper. Using the scraper, it is possible to extract Spotify track information, playlist information, download cover art of Spotify tracks or download mp3 previews of Spotify tracks. This did, however, also mean it was impossible to scrape information from albums or individual artists, which would be more useful for our research.
The next option was using Spotipy, a lightweight Python library for the Spotify Web API. Installing this package was again done by using Pycharm, but there were still a couple steps before I could use the code. Firstly, a Spotify account is needed to access Spotify for Developers to set up an app using My Dashboard. There are two ways to get the proper user authorization, Authorization Code Flow and Client Credentials flow. The first one is suitable for long-running applications which the user logs into once. It provides an access token that can be refreshed, but you have to add a redirect URI to your app, this was fairly simple as the site used some examples that could be used. The second one makes it possible to authenticate your requests to the Spotify Web API and to obtain a higher rate limit than you would with the Authorization Code flow.
The next step was to use the code shown on the website, but there were still several parts of the code I didn’t understand how to use such as the “scopes'' and the “Spotify URI’s and ID’s”. Scopes are useful for the user itself, it can give information such as the users recently played tracks or the users follow count, but since I wanted general information, not specific to the user alone, these scopes were not needed. The website defined a Spotify URI as “The resource identifier that you can enter, for example, in the Spotify Desktop client’s search box to locate an artist, album, or track.” A Spotify ID is “A base-62 number that you can find at the end of the Spotify URI (see above) for an artist, track, album, etc.” The way to obtain the Spotify URI should have been to directly copy it from the Spotify desktop app, but this method did not work for me, so I used the Spotify ID instead. This required a bit more work as you had to copy the artist link and then extract the Spotify ID from that link, for example: if the artist link is “https://open.spotify.com/artist/6pNgnvzBa6Bthsv8SrZJYl?si=nxQ9yxfuRv2i0adrarg2uQ ”, the underlined part after the “artist/” and before the “?” would be the ID. After obtaining all the necessary information to use the code and attempting to get a result, I kept getting errors without being able to find out the source. So I decided to go another route.
import spotipy
from spotipy.oauth2 import SpotifyClientCredentials
auth_manager = SpotifyClientCredentials()
sp = spotipy.Spotify(auth_manager=auth_manager)
playlists = sp.user_playlists('spotify')
while playlists:
for i, playlist in enumerate(playlists['items']):
print("_%4d_ _%s_ _%s_" % (i + 1 + playlists['offset'], playlist['uri'], playlist['name']))
if playlists['next']:
playlists = sp.next(playlists)
else:
playlists = None
The Spotify for Developers website has a multitude of features including guides for the web API, IOS and Android guides and design guidelines. It also includes a console which is much more user friendly and makes it easier to not make any mistakes. Using this console, you can get information on albums, artists, tracks, playlists, shows and much more. If you want information of, for example, an artist, you still need to get the proper authorization by using the Authorization Code Flow or Client Credentials Flow mentioned above and the Spotify ID. The different functions can have different requirements such as market, country_id, additional_types, device_id, time_range, offset, limit and much more.
The result is information in a JSON format, which also includes various non-relevant information such as the dimensions of the picture of the artist. We can put this into a JSON converter to be able to have a more readable result. We can now easily view information such as the amount of followers, the genres, the name of the artist, and also a popularity score, this score is received from the Spotify API. The value will be between 0 and 100, with 100 being the most popular. The popularity is calculated by algorithm and is based, in the most part, on the total number of plays the track has had and how recent those plays are.
{
"external_urls":{
"spotify":"https://open.spotify.com/artist/6pNgnvzBa6Bthsv8SrZJYl"
},
"followers":{
"href":null,
"total":606395
},
"genres":[
"vocaloid"
],
"href":"https://api.spotify.com/v1/artists/6pNgnvzBa6Bthsv8SrZJYl",
"id":"6pNgnvzBa6Bthsv8SrZJYl",
"images":[],
"name":"Miku Hatsune",
"popularity":68,
"type":"artist",
"uri":"spotify:artist:6pNgnvzBa6Bthsv8SrZJYl"
}
All these various methods and websites required some form of basic Python knowledge. So I used websites like w3schools.com and Python.org, as well as the websites for the scrapers themselves to gain a very basic understanding of Python needed to be able to use the programs.
Vocadb.net scraping
To extract data from the Vocadb.net database I stumbled upon a very handy tool called Scraper.AI webplugin Roadwork.io. Instead of writing complex code using python you can just hover your mouse over the part you want to extract data from. You have the possibility to extract single or multiple cells. Because I wanted to make a comparison at how Vocaloids evolved through the years in terms of popularity I extracted data from the overall ranking and a monthly ranking from april to compare how vocaloid songs from 2021 compete against songs from the last decade.
Cleaning with Openrefine
The data I extracted from the Vocaloid database was already very clean so not much changes were made to the dataset. I only used a simple GREL command on both monthly and overall datasets to remove unnecessary words in the cells. I also sorted the ranking so we had the most popular one on top. I extracted this final dataset to work further with in Tableau.
Tableau
I had a completely different view on how to visualize the data we extracted from VocaDb but in the end it didn’t go how I wanted it to so instead of a pie/bar graph I ended with the 2 following visualized results which still presents a decent result on how vocaloid content from this month on Nico Nico Douga compares to overall content of the the last decade.
Conclusion
Although Vocaloid has experienced a decline in overall popularity in the last decade we can't deny from the results of these charts that Hatsune Miku still, after 10 years, not only scores the top ranking with songs from her but all kinds of content surrounding her based on the videos from Nico Nico Douga which is obviously a huge accomplishment. While her fellow vocaloids are slowly fading away, she also has another resurgence in 2021 with a new upcoming live concert made together with fans meaning we are most certainly not yet rid of the Miku craze for the upcoming years.
Song breakdown
Introduction of the software
After all this reading regarding Vocaloid and seeing the results from our research, you might have asked yourself, "How are these voices actually made?" If not, no worries I am here to clarify and simplify this for everyone anyway. When I was first thinking about a possible topic, I wondered how I could possibly combine my passion for music production and interest in Japanese culture because there is nothing that motivates me more than combining my 'best of both worlds'. After thinking for a while, it occurred to me that despite a general decrease in popularity over the years, Vocaloids were still a hit in Japan and could possibly be worked with as a topic.
I already knew Hatsune Miku and Rin Kagamine from videos that went viral over the internet like the "PoPiPo" song by Hatsune Miku for example, but I never knew who the creators behind this were and how on earth they managed to let those virtual characters sing. Well, since I’ve started electronic music production in 2018 & also by having some hardware synthesizers at home I already had quite some knowledge of synthesizers and their usage so the answer was actually quite simple: Hatsune, Rin and other Vocaloids are actually a kind of 'voicebank' as Yamaha calls it and are created to work inside the program: VOCALOID.
This is a program that is used to synthesize a singing voice. And is thus very close to using a normal synthesizer. But where with synthesizers you generate sound through different waveforms, here you generate a real voice which is recorded by a Japanese 声優 or voice actress in advance and which is then further processed in a voice bank. The program component that handles the signal processing was developed by Kenmochi Hideki who is considered the "father" of Vocaloid. The software thus allows users to synthesize human voice by entering text on a sequencer that determines the pitch of the voice just as if you were actually playing an instrument.
Song Breakdown: VOCALOID 5 + Ableton
Therefore, out of my own interest, I thought it would be super interesting to make a song with this as I personally felt it would add a lot of value to our project.
The program I made the song in is called Ableton. It is my go to program for music production and it is also called a DAW or a Digital Audio Workstation. A very versatile program which combines the flexibility of a sequencer that can record different sorts of audio & MIDI with live performance thus why it is called Ableton Live. But before I dived into making the song I had to figure out how VOCALOID 5 operated, to synthesize the vocals & obviously what kind of elements make up a ‘vocaloid song’? After listening to some of the most popular songs I saw a returning pattern and a common theme in all of the songs. A simple groove accompanied by a catchy melody, thick and crunchy bassline, some background pads for creating an ethereal atmosphere, cute random sound FX and of course the synthesized vocals. The voicebank I will be working with for this project is called Yuzuki Yukari. She is the first voicebank developed for VOCALOID3 and the first Vocaloid to also have a Voiceroid component, able to synthesize speech as well as singing. So after getting a grasp of the general idea i was going for i began looking up YouTube tutorials on how to work with the VOCALOID 5 software and through some trial-and-error circumstances I more or less started to understand the program. I started writing a simple Japanese sentence which then by choosing the right pitches despite the 'robotic voice started to sound very musical. I am not specialized in writing song lyrics but in the video below you can see how I created the first sentence of the song in the software.
Then for some padding in my track I started writing some aaaaa's and ooooh's and learned through another tutorial that you can give a very human-like touch to the voice through the 'Enka' option in your editing tools. You can see this in the video below. The same process is applied for all the other vocals or random chops you will hear throughout the song.
In the last video I am going to give you a little glimpse into what my finished music project looks like and how everything came together. I am not going to go into the whole mixing process because that is a completely different and more complex story but I'm just going to show you the puzzle blocks that make up the final track. On the right side of the video, you can see all the elements grouped by name. If you watch the video below you will first hear a melody and chord progression. I wrote this right after the Vocaloid sequence because when you work with voices in a song, melody and chords must complement the singer. If these work together nicely in harmony you have a very nice foundation for the rest of a track. I chose to do this first and then add a bass line because it is easier for me to jam this way but that's different for every producer’s workflow. Throughout the rest of the video you will hear new elements including my drums and percussion that set the whole rhythm of the track which is as equally important as all the rest of the track and then eventually you will hear everything together into a nice cohesive whole.