The Next Generation Poetic Experience

 

Author(s):


Exhibition:


Category:


Abstract/Summary/Introduction:

  • This paper presents the motivation, background and implementation of The Muses of Poetry, an interactive installation that combines dynamically generated character animation, semantic analysis, natural voice interaction and affect in poetry. Inspired by the subjectivity and ethereal quality of this literary art, we wanted to enhance the act of reciting poetry by providing a set of characters the possibility to “understand” and manifest the emotional content of the poems through facial expressions and affective speech. We believe that this original installation will bring poetry closer to a wider audience, while creating a playful, interactive and surprising experience for the user.

References:

  • BEE, N., ANDR´ E, E., AND TOBER, S. 2009. Breaking the ice in human-agent communication: Eye-gaze based initiation of contact with an embodied conversational agent. In Proceedings of the 9th International Conference on Intelligent Virtual Agents, IVA ’09, 229–242.

    BLACK, A., BUNNELL, H., DOU, Y., KUMAR MUTHUKUMAR, P., METZE, F., PERRY, D., POLZEHL, T., PRAHALLAD, K., STEIDL, S., AND VAUGHN, C. 2012. Articulatory features for expressive speech synthesis. In Acoustics, Speech and Signal Processing (ICASSP), 2012 IEEE International Conference on, 4005–4008.

    COLTON, S., GOODWIN, J., AND VEALE, T. 2012. Full face poetry generation. In Proceedings of the Third International Conference on Computational Creativity, 95–102.

    COPE, D. H. 2011. Comes the Fiery Night. CreateSpace Independent Publishing Platform.

    COURGEON, M., BUISINE, S., AND MARTIN, J.-C. 2009. Impact of expressive wrinkles on perception of a virtual character’s facial expressions of emotions. In Proceedings of the 9th InternationalConferenceonIntelligentVirtualAgents,Springer-Verlag, Berlin, Heidelberg, IVA ’09, 201–214.

    COWIE, R., DOUGLAS-COWIE, E., TSAPATSOULIS, N., VOTSIS, G., KOLLIAS, S., FELLENZ, W., AND TAYLOR, J. 2001. Emotion recognition in human computer interaction. IEEE Signal Processing Magazine, 32–80.

    DUHAMEL, P., AND WHISSELL, C.,1998. Thedictionaryofaffect in language [computer software]. FISHER, C. 2009. Andromeda. Electronic Literature Collection, Volume Two.

    GERVAS, P., HERV’AS, R., AND ROBINSON, J. R. 2007. Difficulties and challenges in automatic poem generation: Five years of research at ucm.

    GREENE, E., BODRUMLU, T., AND KNIGHT, K. 2010. Automatic analysis of rhythmic poetry with applications to generation and translation. In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, Association for ComputationalLinguistics,Stroudsburg,PA,USA,EMNLP’10, 524–533.

    HELZLE, V., SPIELMANN, S., AND ZWEILING, N. 2011. Emote, a new way of creating animated messages for web enabled devices. In Proceedings of CVMP 2011.

    KASAP, Z., MOUSSA, M. B., CHAUDHURI, P., AND MAGNENATTHALMANN, N. 2009. Making them remember – emotional virtual characters with memory. IEEE Computer Graphics and Applications 29, 2, 20–29.

    KWIATEK, K., AND WOOLNER, M. 2010. Let me understand the poetry. embedding interactive storytelling within panoramic virtual environments. In EVA 2010, 199–205.

    NIELSEN, F. A. 2011. A new anew: evaluation of a word list for sentiment analysis in microblogs. In Proceedings of the ESWC2011 Workshop on ’Making Sense of Microposts’: Big things come in small packages, M. Rowe, M. Stankovic, A.-S. Dadzie, and M. Hardey, Eds., vol. 718 of CEUR Workshop Proceedings, 93–98.

    NIEWIADOMSKI, R., BEVACQUA, E., MANCINI, M., AND PELACHAUD, C. 2009. Greta: an interactive expressive eca system. In Proceedings of The 8th International Conference on AutonomousAgentsandMultiagentSystems-Volume2,International Foundation for Autonomous Agents and Multiagent Systems, Richland, SC, AAMAS ’09, 1399–1400.

    PFEIFFER, T., LIGUDA, C., WACHSMUTH, I., AND STEIN, S. 2011. Living with a virtual agent: Seven years with an embodied conversational agent at the heinz nixdorf museumsforum. In Proceedings of the Re-Thinking Technology in Museums 2011 Emerging Experiences, thinkkcreative&theUniversityofLimerick, S. Barbieri, K. Scott, and L. Ciolfi, Eds., 121–131.

    RAUH, R., AND SCHALLER, U. M. 2009. Categorical perception of emotional facial expressions in video clips with natural and artificial actors: A pilot study. Tech. rep., Freiburg: Abt. fr Psychiatrie und Psychotherapie im Kindes- und Jugendalter.

    REITHINGER, N., GEBHARD, P., L¨OCKELT, M., NDIAYE, A., PFLEGER, N., AND KLESEN, M. 2006. Virtualhuman: dialogic and affective interaction with virtual characters. In Proceedings of the 8th international conference on Multimodal interfaces, ACM, New York, NY, USA, ICMI ’06, 51–58.

    SCHRODER, M. 2009. Expressive speech synthesis: Past, present, andpossiblefutures. AffectiveInformationProcessing,111–126.

    SIMANOWSKI, R. 2011. Digital Art and Meaning: Reading Kinetic Poetry, Text Machines, Mapping Art, and Interactive Installations. University of Minnesota Press.

    SMALL DESIGN FIRM INC, 2004. L’or´eal poetry harp. Presented in Cambridge, USA.

    STERN, A. 2003. Creating Emotional Relationships with Virtual Characters. Ed. R. Trappl, P. Petta, and S. Payr, MIT PressMIT Press.

    STRAPPARAVA, C., AND VALITUTTI, A. 2004. Wordnet-affect: an affective extension of wordnet. In 4th International Conference on Language Resources and Evaluation (LREC 2004), 1083–1086.

    TOSA, N., AND NAKATSU, R. 1998. Interactive poem system. In Proceedings of SIGGRAPH 98, Annual Conference Series, 115– 118.

    UTTERBACK, C., AND ACHITUV, R., 1999. Text rain. Permanent exhibition at the 21c Museum Hotel, Louisville, USA.