Introductionto Multimedia18Chapter1and hypermedia.For more advanced reading,the collection of surveypapers by Jeffay andZhang [1] provides in-depth background as well as future directions of research.Otherlinks in thetextwebsite include informationon. Ted Nelson and the Xanadu project.Nicholas Negroponte's work at the MIT Media Lab. Negroponte's small book onmultimedia [7] hasbecome a much-quoted classic..Douglas Engelbart and the history of the "On-Line System"。The MIT Media Lab.Negroponte and Wiesner cofounded theMIT Media Lab, whichis still going strong and is arguably the most influential idea factory in the world.Client-side execution.Java and client-side execution started in 1995;"Duke",thefirst JAVA applet, is also on the textbook's web site.Chapter12ofBuford's book[8]provides adetailed introduction to authoring.Neuschotz'sintroductory text [9] gives step-by-step instructions for creating simple Lingo-based inter-active Directormovies.OtherlinksincludeDigital Audio.This web page includes a link to the Sonic Foundry company forinformation on Sound Forge, a sample Sound Forge file, and the resulting outputWAVfile.The example combines left and right channel information in a complexfashion.Little effort is required to produce sophisticated special effects with this tool.Digidesign is one firm offering high-end Macintoshsoftware,whichcan even involvepurchasing extra boards for specialized processing.Music sequencing and notationGraphics and image editing informationVideo editingproducts and information。Animation sites.Multimedia authoring tools。XML:1.6EXERCISES1.Identify three novel applications of the Internet or multimedia applications.Discusswhy you think these are novel.2.Briefly explain, in your own words,"Memex"and its role regarding hypertext, Couldwe carry out the Memex task today? How do you use Memex ideas in your own work?
Section1.7References193.Your task is to think about the transmission of smell over the Internet. Suppose wehave a smell sensor at one location and wish to transmit the Aroma Vector (say) toa receiver to reproduce the same sensation. You are asked to design such a system.List threekey issues to consider and two applications of such a delivery system.Hint:Think about medical applications.4.Tracking objects orpeoplecan bedonebyboth sight andsound.While vision systemsare precise, they arerelatively expensive; on the other hand, a pair ofmicrophones candetect a person's bearing inaccurately but cheaply.Sensorfusion of sound and visionis thus useful. Surf the web tofind out who is developing tools for video conferencingusing this kind of multimedia idea.5.Non-photorealistic graphics means computergraphics that do well enough withoutattempting to make images that look like camera images.An example is conferenc-ing (let's look at this cutting-edge application again).For example, if we track lipmovements, we can generate the right animation to fit our face. If we don't muchlikeourownface,we can substituteanother one-facial-featuremodelingcan mapcorrect lip movements onto another model. See if you can find out who is carryingout research on generating avatars to represent conference participants'bodies.6.Watermarking is a means of embedding a hidden message in data.This could haveimportant legal implications:Is this image copied? Is this imagedoctored?Whotook it? Where? Think of"messages" that could be sensed while capturing an imageand secretly embedded in the image, so as to answer these questions. (A similarquestion derives from the use of cell phones. What could we use to determine whois putting this phone to use,andwhere,and when?This could eliminate the needforpasswords.)1.7REFERENCES1K.JeffayandH.Zhang,Readings inMultimediaComputing and Networking,SanFrancisco:Morgan Kaufmann, CA,2002.2Vannevar Bush,"As We May Think,"The Atlantic Monthly, Jul.1945.3 D.Engelbart and H. Lehtman, Working Together, BYTE Magazine, Dec. 1998, 245-252.4 N. Yankelovitch, N. Meyrowitz, and A. van Dam,"Reading and Writing the Electronic Book,"in Hypermedia and Literary Studies, ed.P.Delany and G.P.Landow, Cambridge, MA:MITPress, 1991.5R.Steinmetz andK.Nahrstedt,Multimedia;Computing,CommunicationsandApplicationsUpper Saddle River, NJ:PrenticeHallPTR, 1995,6J.Nielsen,Multimedia and Hypertexr:TheInternet and Beyond,SanDiego:AP Professional1995.7N.Negroponte,Being Digital, New York: Vintage Books, 1995.8 J.FK.Buford,Multimedia Systems, Reading,MA:Addison Wesley,1994.9 N.Neuschotz, Introduction toDirector and Lingo:Multimedia and Internet Applications,Upper Saddle River, NJ: Prentice Hall, 2000
CHAPTER2Multimedia Authoring andTools2.1 MULTIMEDIAAUTHORINGMultimedia authoring is the creation of multimedia productions, sometimes called"movies"or"presentations",Since we areinterested in this subjectfrom a computer science pointof view, we aremostly interested in interactive applications.Also,we need to considerstill-image editors, such as Adobe Photoshop,and simple video editors, such as AdobePremiere,because these applications help us create interactive multimedia projects.How much interaction is necessary or meaningful depends on the application.Thespectrum runs from almost no interactivity,as in a slide show, to full-immersion virtualrealityIn a slide show, interactivity generally consists of being able to control the pace (e.g.click to advance to the next slide). The next level of interactivity is being able to controlthe sequence and choose where to go next.Next is media control: start/stop video, searchtext,scroll the view,zoom.More control isavailable if we can control variables,such aschanging a database search query.The level of control is substantiallyhigher if we can control objectssay,movingobjects around a screen, playing interactive games,and so on.Finally, we can control anentiresimulation:moveourperspectiveinthescene,controlsceneobjects.For some time,people have indeedconsidered what shouldgo into a multimedia project;references are given at the end of this chapter.In this section, we shall look at.Multimedia authoring metaphors.Multimediaproduction.Multimedia presentation。Automatic authoringThe final item deals with general authoring issues and what benefit automated tools, usingsome artificial intelligence techniques, for example, can bring to the authoring task. As afirst step,we consider programs that carry out automatic linking forlegacy documents.Afteran introduction tomultimedia paradigms,we present some ofthe practical toolssoftware tools that form the arsenal of multimediaofmultimediacontentproduction-20
Section2.1Multimedia Authoring21production. Here we go through the nuts and bolts of a number of standard programscurrently in use.2.1.1 Multimedia Authoring MetaphorsAuthoring is the processofcreating multimedia applications.Mostauthoringprograms useone of several authoring metaphors, also known as authoring paradigms: metaphors foreasier understanding of the methodology employed to create multimedia applications [1].Some common authoringmetaphors areas follows:Scripting language metaphorThe idea here is to use a special language to enable interactivity (buttons, mouse, etc.)and allow conditionals,jumps,loops,functions/macros,and so on.An example istheOpenScriptlanguage in AsymetrixLearning Systems'Toolbook program.Open-Script looks like a standard object-oriented, event-driven programming language.Forexample,a small Toolbookprogram is shownbelow.Sucha language has alearningcurve associated with it, as do all authoring tools even those that use the standard Cprogramming language as their scripting languagebecause of the object librariesthat must be learned.load anMPEG fileextFileName of MediaPlayer"theMpegPath"="c:|windows|media|home33.mpg";-playextplayCount of MediaPlayer "theMpegPath" - l;put the Mediaplayer in frames mode (not time mode)extDisplayMode of Mediaplayer "theMpegPath" = 1;- if want to start and end at specific frames:extSelectionstart of Mediaplayer "theMpegPath"=103;extSelectionEnd of Mediaplayer "theMpegPath"=1997;start playback+get extplay() of Mediaplayer"theMpegPath";eSlide show metaphorSlide shows are by default a linear presentation. Although tools exist to performjumps in slide shows,few practitioners use them.Example programs are PowerPointor ImageQ.。Hierarchical metaphorHere, user-controllable elements are organized into a tree structure. Such a metaphorisoftenused inmenu-drivenapplications..Iconic/flow-controlmetaphorGraphical icons are available in a toolbox, and authoring proceeds by creating aflowchart with icons attached.The standard example of such a metaphor is Author-ware, by Macromedia.Figure 2.1 shows an example flowchart.As well as simpleflowchart elements,such as an IF statement,a CASE statement,and so on, we can
22Chapter2MultimediaAuthoringandToolseq4p路国区宁Level1welcomescreenrandomquiz国窗国resutsFIGURE2.1:Authorwareflowchart.group elements using a Map (i.e., a subroutine) icon. With lttle effort, simple ani-mationisalsopossible。FramesmetaphorAs in the iconic/flow-control metaphor,graphical icons are again available in a tool-box, and authoring proceeds by creating a flowchart with icons attached.However,rather than representing the actual flow of the program, links between icons are moreconceptual.Therefore,"frames"of icon designs represent more abstraction than inthe simplericonic/flow-control metaphor.Anexample of such a program is Quest,byAllen Communication.The flowchart consists of"modules" composed of"frames"Frames are constructedfrom objects, such as text, graphics, audio, animations, andvideo, all of which can respond to events.A real benefit is that the scripting languagehere is the widely used programming language C.Figure2.2 shows a Quest frame.e Card/scripting metaphorThis metaphor uses a simple index-card structure to produce multimedia productions.Since links are available, this is an easy route to producing applications that usehypertext or hypermedia. The original of this metaphor was HyperCard by Apple.Another exampleis HyperStudiobyKnowledgeAdventure.The latter program isnow used in many schools.Figure 2.3 shows two cards in a HyperStudio stack.Cast/score/scripting metaphorIn this metaphor, time is shown horizontally in a type of spreadsheet fashion, whererows, or tracks, represent instantiations of characters in a multimedia production.Since these tracks control synchronous behavior, this metaphor somewhat parallels amusic score.Multimedia elements are drawn froma"cast"ofcharacters,and"scripts