Semantic Albuming for Digital Home Photo
2004.03 ~
 
 Recently, digital camera is getting popular more and more as it is getting cheaper, smaller, and easier to use. Many people tend to freely take a great number of photos in anytime and anywhere with their portable digital camera. With the proliferation of the digital camera, the volume of photos is continually increasing. Many people are likely to share events, experiences, and memories on the photos taken with their friends and family. Accordingly, in order to organize and manage of a large number of photo collections in effective and efficient manners, personal digital photo album is much needed.
 
The goal of this project is the followings
 
1. Category-based Home Photo Album
Content-based retrieval and categorization
Low-level feature description and the semantic description
 

Fig. 1. Overall procedure for Category concept learning procedure

 
 Categorization is a useful way to index home photos into meaningful groups such as people, landscape, architecture, indoors, etc. The category-based indexing also reduce search ranges by effective searching and filtering desired photos. The idea is from that objects are positioned in foreground, while landscape in background scenery. With this structural properties of semantic scenes of the photos, we make models for local semantics, and expand them for global semantics. Their combinational learning of them is proposed for photo categorization.
 The overall procedure of the category concept learning is shown in Fig. 1. First, an entire image region is localized by several sub-regions in terms of photographic region template set. Low-level visual features of the local region are used for local concept learning. Once the local concept detector has been made, the confidence vector of the local region is used for global concept learning then the global concept detectors are generated. All the concepts are trained by support vector machine.
 Each local/global concept is trained by support vector machine. Multiple low-level features are input to local concept detector. The output of every local concept detector constructs a confidence vector of local concepts, which is input to global concept detector.
 
2. Situation-based Home Photo Album
Similar background/view scenery
Highly close relevance in time
 

Fig. 2. Overall procedure for Situation-based Home Photo Album

 
 A series of photos can split into several situation groups by considering both visual change and time gap in between photos. For similarity matching to detect situation change boundary, first, a photo sequence is sorted in time accending order. And the time difference is measured by a log-scaled similarity measurement so as to make it less sensitive to the large time difference. With the time gap, the situation similarity between two photos is measured with the content-based features and their importance.
 
3. Person-identity-based Album
To cluster photo based on person identity features
Performance enhancement by using face and appearance information
 

Fig. 3. Overall procedure for Person-identity-based Home Photo Album

 
 A series of photos taken in a situation often contain similar characters as well as similar background. In usual case, the clothes on which the characters put may not be changed at least in a situation. With this observation, the face and cloth information is combined on top of the given situation clusters to identify persons on photos.
 Fig. 3 shows the person identification procedure. Given the situation clusters, two person identity features – face and its corresponding clothes – are extracted from photos. Then, we make person groups with the identity features. We used a agglomative method for this unsupervised clustering. The person group is indexed into a person of database where it is assumed that several persons would be modeled in prior.
 
 
 
 
 
Seungji Yang, Kyong Sok Seo, Sang-Kyun Kim, Yong Man Ro, Person-identity-based Clustering for Digital Home Photo Albuming, Proc. of Workshop on Image analysis for Multimedia Interactive Services, April. 2005
Seungji Yang, Kyong Sok Seo, Yong Man Ro, Sang-Kyun Kim, Ji-Yeon Kim, Yang Suk Seo, User-centric Digital Home Photo Album, ISCE2005, p.226~229 (CD Version. 108), Jun. 2005
Seungji. Yang, Kyong Sok. Seo, Yong Man Ro, Automated situation clustering of home photos for digital albuming, SPIE Electronic Imaging, Vol.5682 p.212~223, Jan. 2005
Seungji Yang, Jeong Hyun Yoon, and Yong Man Ro, Automatic Image Categorization using MPEG-7 Description, SPIE Electronic Imaging, Vol. 5018, Jan. 2003
Seung Ji Yang, Jeong Hyun Yoon, Ho Kyung Kang, and Yong Man Ro, Category Classification using Multiple MPEG-7 Descriptors, CISST2002, vol.1, June, 2002
서경석, 양승지, 김상균, 노용만, 김지연, 서양석, "디지털 사진 앨범을 위한 인물 특징 기반 분류 및 색인", 2005년도 한국멀티미디어학회 춘계 학술 발표대회
Seungji Yang, Kyong Sok Seo, Sang Kyun Kim, Yong Man Ro, "Situation Clustering Using Visual Semantic Hints", 한국멀티미디어학회 춘계학술발표대회
Sang-Kyun Kim, Seungji Yang, Yong Man Ro, Ji-Yeun Kim, "CE Report for Face-based Photo Clustering Experiment (VCE-3)", MPEG-7 Visual Group, ISO/IEC JTC1/SC29/WG11, MPEG05/M11967, 2005. 4
Sang-Kyun Kim, Seungji Yang, Yong Man Ro, Ji-Yeun Kim, "CE report of category-based photo clustering (VCE-2)", MPEG-7 Visual Group, ISO/IEC JTC1/SC29/WG11, MPEG05/M11966, "2005. 4
Sang-Kyun Kim, Seungji Yang, Yong Man Ro, Ji-Yeun Kim, "CE Report on Situation/View-based Photo Clustering Experiment (VCE-1)", MPEG-7 Visual Group, ISO/IEC JTC1/SC29/WG11, MPEG05/M11965, 2005. 4
Sang-Kyun Kim, Seungji Yang, Yong Man Ro, Ji-Yeun Kim, "GT set proposed for person-identity based photo clustering experiment (VCE-3)", MPEG-7 Visual Group, ISO/IEC JTC1/SC29/WG11 M11655, 2005. 1
Sang-Kyun Kim, Seungji Yang, Yong Man Ro, Ji-Yeun Kim, "CE Report for Face-based Photo Clustering Experiment (VCE-3)", MPEG-7 Visual Group, ISO/IEC JTC1/SC29/WG11 M11653, 2005. 1
Sang-Kyun Kim, Seungji Yang, Yong Man Ro, Ji-Yeun Kim, "GT set and model set proposed for category based photo classification experiment (VCE-2)",MPEG-7 Visual Group, ISO/IEC JTC1/SC29/WG11 M11652, 2005. 1
Sang-Kyun Kim, Seungji Yang, Yong Man Ro, Ji-Yeun Kim, "CE Report for Category-based Photo Clustering Experiment (VCE-2)", MPEG-7 Visual Group, ISO/IEC JTC1/SC29/WG11 M11650, 2005. 1
Sang-Kyun Kim, Seungji Yang, Yong Man Ro, Ji-Yeun Kim, "GT set proposed for situation/view based photo clustering experiment (VCE-1)", MPEG-7 Visual Group, ISO/IEC JTC1/SC29/WG11 M11648, 2005. 1
 Sang-Kyun Kim, Seungji Yang, Yong Man Ro, Ji-Yeun Kim, "Photo contribution status for face-based photo clustering experiment (VCE-3)", MPEG-7 Visual Group, ISO/IEC JTC1/SC29/WG11 M11304, 2004. 10
Sang-Kyun Kim, Seungji Yang, Kyeong Seok Seo, Yong Man Ro, Ji-Yeun Kim, "CE Report for Face-based Photo Clustering Experiment (VCE-3)", MPEG-7 Visual Group, ISO/IEC JTC1/SC29/WG11 M11303, 2004. 10
Sang-Kyun Kim, Seungji Yang, Yong Man Ro, Ji-Yeun Kim, "CE Report for Category-based Photo Clustering Experiment (VCE-2)", MPEG-7 Visual Group, ISO/IEC JTC1/SC29/WG11 M11301, 2004. 10
Sang-Kyun Kim, Seungji Yang, Yong Man Ro, Ji-Yeun Kim, "Photo contribution status for face-based photo clustering experiment (VCE-3)", MPEG-7 Visual Group, ISO/IEC JTC1/SC29/WG11 M11304, 2004. 10
Sang-Kyun Kim, Seungji Yang, Yong Man Ro, Ji-Yeun Kim, "GT set proposed for category based photo classification experiment (VCE-2)", MPEG-7 Visual Group, ISO/IEC JTC1/SC29/WG11 M1130 , 2004. 10
 Sang-Kyun Kim, Seungji Yang, Yong Man Ro, Ji-Yeun Kim, "A proposal of GT set determination where ‘Time’ and ‘Visual’ information conflict for Situation/View based photo clustering (VCE-1)", ISO/IEC JTC1/SC29/WG11 M1130, 2004. 10
Sang-Kyun Kim, Seungji Yang, Yong Man Ro, Ji-Yeun Kim, "Photo EXIF modification report on Ground Truth set of Situation/View based photo clustering CE (VCE-1)", ISO/IEC JTC1/SC29/WG11 M11299, 2004. 10
Sang-Kyun Kim, Seungji Yang, Yong Man Ro, Ji-Yeun Kim, "GT set proposed for situation/view based photo clustering experiment (VCE-1)" , ISO/IEC JTC1/SC29/WG11 M11298, 2004.10
Sang-Kyun Kim, Seungji Yang, Yong Man Ro, Ji-Yeun Kim, "CE Report for Situation/View-based Photo Clustering Experiment (VCE-1)", ISO/IEC JTC1/SC29/WG11 M11297, 2004. 10
Sang-Kyun Kim, Seungji Yang, Yong Man Ro, Ji-Yeun Kim, "Consideration of face based photo clustering and retrieval", ISO/IEC JTC1/SC29/WG11 M10953, 2004. 7
Sang-Kyun Kim, Seungji Yang, Yong Man Ro, Ji-Yeun Kim, "Model set and GT set for the photo categorization experiment (VCE-2)", ISO/IEC JTC1/SC29/WG11 M10951, 2004. 7
Sang-Kyun Kim, Seungji Yang, Yong Man Ro, Ji-Yeun Kim, "Consideration on the photo categorization scheme (VCE-2)", ISO/IEC JTC1/SC29/WG11 M10950, 2004. 7
Sang-Kyun Kim, Seungji Yang, Yong Man Ro, "GT set proposed for situation/view based photo clustering experiment (VCE-1)", ISO/IEC JTC1/SC29/WG11 M10949, 2004. 7
Sang-Kyun Kim, Seungji Yang, Yong Man Ro, Ji-Yeun Kim, "CE report for situation/view based photo clustering experiment (VCE-1)", ISO/IEC JTC1/SC29/WG11 M10948, 2004. 7
Sang-Kyun Kim, Seungji Yang, Yong Man Ro, Ji-Yeun Kim, "Consideration on the clustering evaluation method for situation/view based photo clustering experiment (VCE-1)", ISO/IEC JTC1/SC29/WG11 M10947, 2004. 7
Sang-Kyun Kim, Seungji Yang, Yong Man Ro, "Initial report on Visual tools and DS for images or photos libraries (VCE-1)", ISO/IEC JTC1/SC29/WG11 M10640, 2004. 3
김상균, 김지연, 노용만, 양승지, "미디어 앨범화 힌트 정보를 이용한 멀티미디어 컨텐츠의 클러스터링/인덱싱 방법 및 시스템(Method and apparatus for clustering and indexing of multimedia contents using media albuming hints)",