▒ ▒ ▒ Project Demo : SmarTV ▒ ▒ ▒

	Intelligent Agent and Metadata Management Technology

	2002 . 01 ~ current

SmarTV (Super intelligent Multimedia Anytime anywhere Realistic TV), next generation intelligent digital broadcasting technology through integration between broadcasting and communication, can make user enjoy desirous programs and broadcasting information, anytime-anywhere and realistically. Intelligent Agent and metadata management technology is the core technology of SmarTV

The goal of this project is the followings

1. Intelligent Broadcasting Platform Development based on Agents

	Broadcasting Agent platform development , compliant FIPA (The Foundation for Intelligent Physical Agents)
	Scenario development for Intelligent broadcasting

2. Intelligent Metadata Management Technology Development

	use TV-Anytime Forum and MPEG-7 metadata

Fig. 1. Intelligent broadcasting system based on Agent Platform

As shown in Fig. 1, Intelligent Broadcasting Platform works through IP Network. Service Provider (SP) broadcasts the content for push & pull service. User Terminal make user require the desirous content through the return channel. Service Information Provider (SIP) has resolution information of SP and profile of Users. Additionally there are EPG server and Video Streaming Server.

Fig 2 shows how many agents each platform is composed of and how agents work one another in each platform and between platforms. Agents work automatically to response user request and to analyze the user preferences and to manage each DB that User Terminal and Service Provider have and so on.

Fig. 2. Agents Working Flow in each Platform

The functions of a few agents are the followings

User Interface Agent :

	Provide user with the interface
	Deliver the information of user's requests to User Profile Agent
	Display the received information or contents

Terminal Agent :

Communication with external Platforms

Location Resolve Agent :

	Communication with external Platforms
	Retrieve the Location Resolution of Service Provider
	Manage the profile received from User Terminal

Provider Agent :

Communication with external Platforms

Management Agent :

	Communication with external Platforms
	Retrieve the Location Resolution of Service Provide

3. Real-Time Content Filtering for TV Terminals

For live broadcasts, a TV terminal such as a set-top box (STB) and a personal video recorder (PVR) needs real-time content filtering technology which segments broadcasts content-levelly. At the same time, it requires the effective management of the filtered video segments about the live broadcasts
In order to propose new video archiving formats for the filtered broadcasting media, we utilize our previous work of the real-time content filtering in a TV terminal. By the proposed format, we expect effective content-based retrieval or browsing of the video segments in a home terminal, e.g., a STB or a PVR, and compatibility in a portable terminal, e.g., a PDA or a mobile device. The usefulness of the proposed archiving method is verified by implementing and analyzing an archiving system and formats about the filtered video segments.

Fig. 3. The procedure of the real-time filtering	Fig. 4. The screen shot to run the real-time content filtering service with a single channel of interest

4. Video Archiving Formats

The main aim of Video Archiving Format is to enable to use auxiliary information with original broadcasting content anytime and anywhere. To achieve it, we adopt a MPEG-21 file format, which can contain various resources and provide easy access or export embedded resources. And also, we utilize TV-Anytime meta-data for describing broadcasting contents and illustrating the structure of related media resources. To verify the usefulness of the proposed format, we introduce a service scenario for multimedia packaging and redistribution in TV terminals, which is suitable to TVAF environment, and test it with music broadcasts and an implemented player.

Requirement :

	It should be able to contain the various multimedia resources.
	It should be easy to access to embedded resources within integrated media.
	It should be able to include multiple A/V streams for various applications, e.g., video album.

Fig. 5. File Structure (1) for video album and (2) for archiving

Fig. 6. TV-Anytime Package description

Its’ Package description, which borrowed from the multi-level structure of MPEG-21 DID, supports to describe the composition of content-related media resources. For example, after recording a music broadcast, the mechanism of TVAF enables to download the music broadcast-related media resources such as singers’ album information, songs’ lyrics, etc. Therefore, a binding method for the downloaded resources and original content is proposed to manage efficiently and effectively.

Fig. 7. The implemented Player

It provides (1)meta-data for the video album and (2) contents’ titles, singers, package’s CRIDs, in this case, four packages. When a user selects the one of the four packages, (3) shows the item list in the selected package. And then, (4) components in the selected item are displayed with offset, length and component’s CRID corresponding to media resource in mdat box. (5) are the results of the resources within the components of selected item. We verified to manage various media resources with only one file.

	International Journal
	Published (3)
	Sung Ho Jin, Tae Meon Bae, and Yong Man Ro, “Automatic Video Genre Detection for Content-based Authoring,” LNCS3331, pp. 335-343. Nov. 2004. (SCIE)
	Tae Meon Bae, Sung Ho Jin, Yong Man Ro, “Video Segmentation using Hidden Markov Model with Multimodal Features,” LNCS3115, pp.401-409, July 2004. (SCIE)
	Tae Meon Bae, Cheon Seog Kim, Sung Ho Jin, Ki Hyun Kim, and Yong Man Ro, “Semantic Event Detection by HMM/SVM Approach in Structured Video,” LNCS3568, pp.113-122, July 2005. (SCIE)
	Accepted (2)
	Sung Ho Jin, Jun Ho Cho, and Yong Man Ro, “Meaningful Scene Filtering for TV Terminals,” IEEE Transaction on Consumer Electronics (published in Feb. 2006) (SCI)
	Sung Ho Jin, Tae Meon Bae, Yong Man Ro, Hoi-Rin Kim, and Munchurl Kim, “Intelligent Broadcasting System and Services for Personalized Semantic Contents Consumption,” Expert system with applications, Elsevier (accepted) (SCIE)
	Sun Ho Jin, Jun Ho Cho, and Yong Man Ro, “A New Real Time Filtering for TV Terminals,” Multimedia Tools and Applications, Springer (submitted) (SCIE)
	Submitted (2)
	Sung Ho Jin, Jun Ho Cho, and Yong Man Ro, “Multimedia Packaging for TVAF-based Broadcasting Content,” LNCS (WIAMIS2006) (submitted) (SCIE)
	Sung Ho Jin, Tae Meon Bae, and Yong Man Ro, "Automatic Video Genre Detection for Content-based Authoring;, LNCS 3331 (SCIE), Nov. 2004.
	International Conference
	Published (9)
	Sung Ho Jin, Tae Meon Bae, Jin Ho Choo, and Yong Man Ro, "Video Genre Classification Using Multimodal Features," Proc. of SPIE2004, Vol.5307, pp.307-318, Jan. 2004.
	Tae Meon Bae, Sung Ho Jin, Jin Ho Choo, Mansoo Park, Yong Man Ro, Hoi-Rin Kim, and Kyeongok Kang, "Video Segmentation by Hidden Markov Model Using Multimodal MPEG-7 descriptors," Proc. of SPIE2004, Vol.5304, pp.2004. Jan. 2004.
	Sung Ho Jin, Tae Meon Bae, Jin Ho Choo, Yong Man Ro and Kyeongok Kang, "A MPEG-7 and TV-Anytime based Multi-Agents Platform for Interactive Broadcasting System,” Proceeding on SPIE Electronic Imaging 2003, Vol , pp. Jan. 2003
	Sung Ho Jin, Tae Meon Bae, Yong Man Ro, and Kyeongok Kang, “Intelligent Agent-based System for Personalized Broadcasting Services,” Proceeding of the International Conference On Image Science, Systems and Technology CISST’04, pp. 613-619, June 2004.
	Cheon Seog Kim, Tae Meon Bae, Jin Ho Choo, Sung Ho Jin, and Yong Man Ro, “Semantic Event Detection in Sports Video Using Hidden Marcov Model,” Proceeding of the International Conference On Image Science, Systems and Technology CISST’04, pp. 585-591, June 2004.
	Sung Ho Jin, Jun Ho Cho, Tae Meon Bae , Yong Man Ro, and Kyeongok Kang , “Content Filtering for Broadcasting Media” SPIE2005 (Internet Imaging), vol. 5670, pp 261-270, Jan. 2005.
	Sung Ho Jin, Jun Ho Cho, and Yong Man Ro, “Real-time Filtering System on Content Semantics for Live Broadcasting,” International Symposium on Consumer Electronics 2005, pp 98-, June 2005.
	Sung Ho Jin, Jun Ho Cho, Yong Man Ro, and Jae-Gon Kim, “Meaningful Scene Filtering for TV terminals,” International Conference on Consumer Electronics, pp. 2006
	Sung Ho Jin, Jung Ho Cho, Yong Man Ro, and Jae-Gon Kim, “Personalized Video Album for TV terminals,” SPIE2006
	Domestic Journal
	Published (3)
	Sung Ho Jin, Tae Meon Bae, Jun Ho Cho, Yong Man Ro and Kyeongok Kang, “User-Centric Broadcasting Environment Using Intelligent Agents,” Journal of Korea Multimedia Society, Vol. 8, No.6, pp. 814-826, June 2005.
	진성호, 조준호, 노용만, 김재곤, “개인화된 의미 기반 콘텐츠 소비를 위한 지능형 방송 시스템과 서비스,” 방송공학회 논문지, vol.10, no.3, pp. 422- 435, 2005
	조준호, 양승지, 진성호, 노용만, 김상균, “사진앨범을 위한 MAF 기반 통합 미디어에 관한 연구,” 방송공학회 논문지, vol.10, no.3, pp. 436-450, 2005
	Submitted (1)
	조준호, 진성호, 이한규, 노용만, “TV-Anytime 기반 방송 콘텐츠를 위한 복합미디어에 관한 연구,” 방송공학회 논문지
	Domestic Conference
	Published (9)
	진성호, 배태면, 추진호, 노용만, 강경옥, "멀티모달 특징을 이용한 비디오 장르 분류", 2003년도 한국방송공학회 정기총회 및 학술대회, pp. 219-222
	추진호, 배태면, 진성호, 강경옥, 노용만, "방송 환경에서 내용기반 검색 서비스에 관한 연구", 2003년도 전자공학회 하계종합학술대회, Vol.26, No.1, pp.1511-1514
	진성호, 배태면, 노용만, 김문철, 강경옥, "지능형 방송 서비스를 위한 멀티 에이전트 시스템 설계에 관한 연구", 2002년도 멀티미디어 추계학술발표대회, Vol.5 No.2 pp. 404-407, 2002.11
	진성호, 배태면, 강석준, 노용만, 강경옥, “멀티모달 특징을 이용한 컨텐츠 기반 저작을 위한 비디오 장르 식별”, 2004년도 전자공학회 하계학술대회, 논문집 27권 1호, pp. 753-756.
	진성호, 조준호, 배태면, 노용만, 강경옥, “다채널 환경에서 컨텐츠 필터링 알고리즘”, 2004년도 신호처리학술합동대회 논문집, pp. 68. 2004-11
	진성호, 조준호, 배태면, 노용만, 강경옥, 홍진우, “개인화된 비디오 요약 방송 서비스”, 2004년 한국방송공학회 학술대회 논문집, pp. 143-146, 2004-11
	조준호, 진성호, 노용만, 강경옥, “실시간 필터링을 위한 축구 비디오의 장면 검출”, 2004년 한국방송공학회 학술대회, pp. 229-232. 2004-11
	조준호, 진성호, 양승지, 노용만, “영상 검색을 위한 MAF 기반 통합 미디어에 관한 연구”, 2005년 멀티미디어학회 춘계학술대회 논문집, pp. 785-788, May 2005.
	진성호, 조준호, 노용만, 이한규, “홈단말에서의 동영상데이터 관리 방법에 대한 연구,” 2005년 멀티미디어학회 추계학술대회
	Domestic Patent
	Registration (1)
	드라마 장르의 비디오 컨텐츠에서 특정인물 등장구간 검출을 위한 오디오 및 비주얼 정로를 이용한 검색 장치 및 방법 {System for Detecting Scenes Containing User Specified Actor Using Audio and Visual Features in Drama Genre Video Contents}, 2002 발명자: 노용만, 배태면, 진성호, 추진호, 강경옥, 이한규, 이희경 등록번호: 0502429
	Application (3)
	디지털 TV 방송의 시청자 맞춤형 비디오 요약 제공 시스템 {An intelligent agent system for providing viewer-customized video skims in digital TV broadcasting}, 2004 발명자: 노용만, 진성호, 배태면, 강경옥, 김진웅, 양승준 출원번호: 10-2004-29359
	디지털 TV 방송에서 지능형 프로그램 필터링 방법 및 에이전트 시스템 {Intelligent program filtering methodology and agents system in digital TV broadcasting}, 2004 발명자: 노용만, 진성호, 배태면, 강경옥, 김진웅, 양승준 출원번호: 10-2004-28308
	비디오 및 메타데이터의 통합을 위한 비디오 멀티미디어 응용 파일 형식의 인코딩/디코딩 방법 및 시스템 {Method and apparatus for encoding and decoding of a video multimedia application format including both video and metadata}, 2005 발명자: 노용만, 진성호, 양승지 출원번호: 10-2005-0088630
	홈단말에서 실시간 필터링된 방송 비디오 관리 시스템 및 그 방법{System for management of real-time filtered broadcasting videos in a home terminal and a method for the same}, 2005 발명자: 노용만, 진성호, 조준호, 윤정현,이한규, 홍진우 출원번호: 10-2005-0099589
	Technical Document
	Published (5)
	지능형 Agent 및 메타데이터 관리 기술 분석서: SMARTV-ITER-TM-011, ETRI
	지능형 Agent 및 메타데이터 관리 요구 사항서: SMARTV-ITER-SR-002, ETRI
	지능형 Agent 및 메타데이터 관리 기능 규격서: SMARTV-ITER-FRS-002, ETRI
	지능형 Agent 및 메타데이터 관리 구조 기능 설계서: SMARTV-ITER-SFD-002, ETRI
	지능형 Agent 및 메타데이터 관리 상세 설계서: SMARTV-ITER-DD-002, ETRI

	You can download a Demo video in a User Terminal which shows personalized broadcasting services using content semantics. Note that because of capturing screen displaying, this video sometimes shows a problem which can not maintain the playing content seamless. It is not system problem, but capturing trouble.

	You can see the recorded video about a prototype system using intelligent agent platform in SmarTV. Note that this video does not include captions.


	Management System for TV terminals This is a demo video for storage management system for TV terminals such as a set-top box and a personal video recorder. In this system, an integrated media format is used which enables to contain broadcasting content and auxiliary information such as meta-data and content-related media. Using the format, the terminals can manage the recorded broadcasting contents efficiently and effectively. The demo video shows that a user can easily find the related information as well as the contents when he searches desired contents in the storage of the terminal. In this demo, music show broadcasts are applied to the system. As additional information about the contents, their meta-data, key-frames, albums’ information, and the lyrics of songs are shown.


	MAF Player for a heterogeneous terminal According to TVAF scenario that MAF-encoded broadcasting contents can be shared or re-distributed between different terminals, the player in this demo is for the terminals which receives the MAF-encoded contents from a home or friend’s terminal. Before sending the recoded contents, a user can combine several broadcasts together by using a single integrated media format, so-called "Video Album." The most benefit of the combined format is to provide story-telling broadcasts or content combination. In this demo, the video album with multiple music show broadcasts is shown.