While visiting Japan last January, I saw a technology demostration that really captured my attention. Its called TVML (TVprogram Making Language) and it could very well change the way we relate to multimedia. It's a script description language being developed by Japanese researchers for use in producing full "TV programs" in real time virtual environments by using computer generated (CG) characters, a voice synthesizer, and familiar multimedia conventions. With this system a user should be able to dictate actions within a virtual space simply by generating a text-based script in real-time. In a TVML script, the contents and actions of a virtual space can be controlled by text-based commands such as "show title#1" or "character walk". The possibility of using voice-to-text capabilities makes this even more interesting. Written or spoken, TVML could be a very compelling way for people to script interactive virtual spaces and simulations using natural laguage approaches. It could also significantly lower the barrier to entry for a good many creative minds and allow for the rapid and low cost development of interactive virtual environments for entertainment, education, and training.
TVML was initially developed by R&D teams at NHK (Japanese Broadcasting Corporation), Hitachi Kokusai Electric Inc., Hitachi, Ltd., Central Research Laboratory, and Keio University. In 2001, the project moved entirely to the NHK where the effort is being led by Hayashi-san and his team and in collaboration with researchers at NICT (The National Institute for Information and Communications Technology). The good news is that they are now working to develop a TVML API and interface module for Croquet!
The first implementation of a TVML-emabled Croquet technology is intended for a Kyoto tourism information and support system. The idea is that a character will respond to tourists questions and guide them to points of interest in a virtual Kyoto. The virtual Kyoto will also be tied in with real-world locations and the technology is therefore being developed for use on multiple real-world display devices. Click here to learn more about TVML.
4 comments:
I think the important part of TVML will prove to be the set of abstractions by which you direct the action. The actual serialization as text will not be as important. In other words, its not the language, but the ideas behind it. You hint at this a little bit with your mention of voice to text.
Consider oddcast.comThe computer generated characters there are directed by a script, as in TVML. Side note: In your blog and the few things I glanced at on http://www.nhk.or.jp, there was no mention of interactivity, though it certainly isn't in conflict with the TVML idea. Oddcast very specifically interacts with the user. For example, notice how the avatars' eyes follow your mouse around when it's done talking. I've seen folks hook these things up to an Eliza-like response loop so that you feel like talking to Max Headroom.
But look at how the customer programs an oddcast. (Go the sitepal store tab.) You pick clothes, hair, gender, race, age, and so forth from a pallet. You pick behaviors from a pallet. You add your voice. I haven't gone through the process, but I would be surprised if you can't make clips that are given in response to various gestures or more complex situations. I bet that the clips can be represented as an iMovie- or Flash-like timeline, with specific output or other state happening at user-directed points. The result of all this could certainly be stored as a human-readable script. That would be good for debugging, for learning tricks from other oddcasts that you like, and for generating scripts by computer programs. But I don't think its a key part of ordinary production. Tim Berners-Lee has said that he didn't imagine folks would write HTML by hand, but that direct-manipulation editors would produce it. Same for TVML.
I went to the TVML web site, read through all the information there, and was excited by what I saw. Then I downloaded the (free!) software and tried it out. All I get is a GDI32.dll error. So, I tried to sign up for their mailing list to see if I could find any support and got no acknowledgement. I tried to just send an email to the list, and got a "No SMTP server at that address" error. Some of this may have to do with the holidays, but so far I'm a bit disappointed!
While looking for further assistance with TVML, I also came across a more current project called "Cyber Theater Scenario Language" and the CTSL Learning Kit, which claims to be more advanced than TVML. This looks very interesting, but as far as I can tell, is just research and not available to the public at this time. daf
I'm Hayashi who made the TVML, mentioned in the Julian's post. Thank you for your introduction, Julian, I'm very glad to read it.
And I'm writing to say sorry to the person of 2nd comment disappointed with the TVML service. I think maybe you accessed and tried the old version of TVML Player & TVML Site. We recently (about a month ago) updated the entire contents of the TVML homepage and also updated the TVML Player itself to the latest version. Please access the site again (http://www.nhk.or.jp/strl/tvml/english/player2/) and download the software and try it. We are sorry for the inconvenience.
Yes, I know CTSL. That's quite similar to TVML. Actually, CTSL came after TVML. It has some functions that TVML doesn't have and vice versa. Differences between two are, I think, output quality (TVML should be better) and availability (CTSL is in house).
Post a Comment