METHODS AND SYSTEMS FOR AGGREGATION AND ORGANIZATION OF MULTIMEDIA DATA ACQUIRED FROM A PLURALITY OF SOURCES

A system for aggregation and organization of multimedia data acquired from a plurality of sources can include a data collection element, a multimedia segmentation element, a memory, an extraction element, and a user interface. The data collection element can be configured to receive multimedia data. The multimedia segmentation element can be coupled to the data collection element and can be configured to separate the multimedia data into a plurality of multimedia data segments. The memory can be configured to store hierarchical category information having a plurality of categories. The extraction element can be coupled to the multimedia segmentation element and the memory and can be configured to associate at least one of the plurality of categories with each of the plurality of multimedia data segments. 1. A system for aggregation and organization of multimedia data acquired from a plurality of sources , comprising:a data collection element configured to receive multimedia data;a multimedia segmentation element, coupled to the data collection element, and configured to separate the multimedia data into a plurality of multimedia data segments;a memory configured to store hierarchical category information having a plurality of categories;an extraction element, coupled to the multimedia segmentation element and the memory, configured to associate at least one of the categories with each of the multimedia data segments; anda user interface, coupled to the extraction element, configured to provide a user with at least one of the multimedia data segments associated with a corresponding one of the categories in the hierarchical category information.2. The system of claim 1 , wherein the data collection element further comprises a recorder configured to record audiovisual data.3. The system of claim 2 , wherein the recorder is configured to record at least one of analog video content and digital video content.4. The system of claim 2 , wherein the recorder further ...

Rapid image annotation via brain state decoding and visual pattern mining

Human visual perception is able to recognize a wide range of targets but has limited throughput. Machine vision can process images at a high speed but suffers from inadequate recognition accuracy of general target classes. Systems and methods are provided that combine the strengths of both systems and improve upon existing multimedia processing systems and methods to provide enhanced multimedia labeling, categorization, searching, and navigation.

Methods and systems for aggregation and organization of multimedia data acquired from a plurality of sources

A system for aggregation and organization of multimedia data acquired from a plurality of sources can include a data collection element, a multimedia segmentation element, a memory, an extraction element, and a user interface. The data collection element can be configured to receive multimedia data. The multimedia segmentation element can be coupled to the data collection element and can be configured to separate the multimedia data into a plurality of multimedia data segments. The memory can be configured to store hierarchical category information having a plurality of categories. The extraction element can be coupled to the multimedia segmentation element and the memory and can be configured to associate at least one of the plurality of categories with each of the plurality of multimedia data segments.

SYSTEM AND METHOD FOR EXTRACTING TEXT CAPTIONS FROM VIDEO AND GENERATING VIDEO SUMMARIES

Caption boxes which are embedded in video content can be located and the text within the caption boxes decoded. Real time processing is enhanced by locating caption box regions in the compressed video domain and performing pixel based processing operations within the region of the video frame in which a caption box is located. The captions boxes are further refined by identifying word regions within the caption boxes and then applying character and word recognition processing to the identified word regions. Domain based models are used to improve text recognition results. The extracted caption box text can be used to detect events of interest in the video content and a semantic model applied to extract a segment of video of the event of interest. 1. A system for generating an event based summary of one or more fields or frames of video content including caption boxes embedded therein , comprising:extracting means for extracting caption boxes from at least a portion of the fields or frames of the video content;change identifying means, coupled to the extracting means and receiving the caption boxes therefrom, for identifying one or more changes in the content of the extracted caption boxes which are indicative of an event of interest; andmodel applying means, coupled to the extracting means and change identifying means and receiving the caption boxes and the one or more changes therefrom, for applying a semantic model to select a portion of the video content preceding each of the one or more changes.2. The system of claim 1 , wherein the video content comprises video of a baseball game claim 1 , and wherein the semantic model comprises a model which identifies the portion of the video content of the event of interest as residing between a pitching event and non-active views.3. The system of claim 2 , wherein the events of interest comprise score events and last pitch events.4. The system of claim 1 , wherein the extracting means further comprises:location means for ...

Multimedia integration description scheme, method and system for MPEG-7

The invention provides a system and method for integrating multimedia descriptions in a way that allows humans, software components or devices to easily identify, represent, manage, retrieve, and categorize the multimedia content. In this manner, a user who may be interested in locating a specific piece of multimedia content from a database, Internet, or broadcast media, for example, may search for and find the multimedia content. In this regard, the invention provides a system and method that receives multimedia content and separates the multimedia content into separate components which are assigned to multimedia categories, such as image, video, audio, synthetic and text. Within each of the multimedia categories, the multimedia content is classified and descriptions of the multimedia content are generated. The descriptions are then formatted, integrated, using a multimedia integration description scheme, and the multimedia integration description is generated for the multimedia content. The multimedia description is then stored into a database. As a result, a user may query a search engine which then retrieves the multimedia content from the database whose integration description matches the query criteria specified by the user. The search engine can then provide the user a useful search result based on the multimedia integration description.

SYSTEMS AND METHODS FOR VISUAL SENTIMENT ANALYSIS

Method for determining one or more viewer affects evoked from visual content using visual sentiment analysis using a correlation model including a plurality of publisher affect concepts correlated with a plurality of viewer affect concepts includes detecting one or more of the plurality of publisher affect concepts present in selected visual content, and determining, using the correlation model, one or more of the plurality of viewer affect concepts corresponding to the one or more of the detected publisher affect concepts. A method for determining one or more visual content to evoke one or more viewer affects using visual sentiment analysis is also provided. 1. A method for determining one or more viewer affects evoked from visual content using visual sentiment analysis using a correlation model including a plurality of publisher affect concepts correlated with a plurality of viewer affect concepts , comprising:detecting, by a processor in communication with the correlation model, one or more of the plurality of publisher affect concepts present in selected visual content; anddetermining, by the processor using the correlation model, one or more of the plurality of viewer affect concepts corresponding to the one or more of the detected publisher affect concepts.2. The method of claim 1 , further comprising providing the correlation model claim 1 , wherein the correlation model comprises a Bayes model to characterize correlations between the plurality of publisher affect concepts and the plurality of viewer affect concepts.3. The method of claim 2 , wherein providing the correlation model further comprises smoothing the correlation model using collaborative filtering.4. The method of claim 1 , further comprising obtaining the plurality of publisher affect concepts claim 1 , wherein the plurality of publisher affect concepts are obtained from metadata associated with visual content in a visual content database.5. The method of claim 1 , further comprising obtaining ...

RAPID IMAGE ANNOTATION VIA BRAIN STATE DECODING AND VISUAL PATTERN MINING

Human visual perception is able to recognize a wide range of targets but has limited throughput. Machine vision can process images at a high speed but suffers from inadequate recognition accuracy of general target classes. Systems and methods are provided that combine the strengths of both systems and improve upon existing multimedia processing systems and methods to provide enhanced multimedia labeling, categorization, searching, and navigation. 1. A computer-based method for labeling multimedia objects comprising:presenting selected multimedia data to a first user;determining user interest in said selected multimedia data based upon brain signal responses of the first user;generating selected multimedia label data corresponding to the selected multimedia data based on the determination of user interest;using a processor and based on at least said generated selected multimedia label data, performing at least one of refining multimedia label data associated with said selected multimedia data or predicting new multimedia label data pertaining to said stored multimedia data by calculating a classification function.2. The method of claim 1 , further comprisingstoring a multimedia affinity graph in one or more memories, wherein said affinity graph represents multimedia data samples as nodes and comprises edges measuring relatedness among data samples; andcalculating a classification function based on at least the selected multimedia label data using a processor associated with said one or more memories, wherein calculating said classification function comprises iteratively performing at least updating selected multimedia label data relating to selected multimedia data or predicting new multimedia label data for stored multimedia data using said processor.3. The method of claim 1 , wherein generating selected multimedia label data comprises decoding the brain signal responses of the user and generating at least one interest score for said selected multimedia data.4. The ...

SYSTEMS AND METHODS FOR IDENTIFICATION OF FLUID AND SUBSTRATE COMPOSITION OR PHYSICO-CHEMICAL PROPERTIES

Techniques for identifying a composition of a target fluid using a set of vectors representing known residue patterns for a two or more fluids including said target fluid is provided. An exemplary method includes storing one or more digital measurements of residue for the target fluid, extracting one or more descriptive features from the measurements; and processing descriptive features to identify the composition of the target fluid. The processing includes using a machine learning algorithm trained with data linking residue morphology to fluid composition. A distance between a vector representing said one or more descriptive features and said set of vectors representing known residue patterns is determined, and a residue is assigned to one or more of the known residue patterns. 1. A method for identifying at least one of a physico-chemical property or composition of a target fluid using a set of vectors representing known residue patterns for a two or more fluids including said target fluid , comprising:acquiring one or more digital measurements of residue for said target fluid;extracting one or more descriptive features from at least one of said measurements; andprocessing one or more of said descriptive features to identify said at least one physico-chemical property or composition of said target fluid from said at least one of said measurements;wherein said processing includes using a matching algorithm or a machine learning algorithm trained with data linking residue patterns to fluid composition or physico-chemical properties.2. The method of claim 1 , further comprising:determining a distance between a vector representing said one or more descriptive features and said set of vectors representing known residue patterns, andassigning a residue to one or more of the known residue patterns that minimizes said distance.3. The method of claim 1 , wherein acquiring one or more digital measurements comprises capturing one or more images.4. The method of claim 1 , ...

SYSTEMS AND METHODS FOR MOBILE SEARCH USING BAG OF HASH BITS AND BOUNDARY RERANKING

Determining ranked candidate media in response to media query data corresponding to a query media includes receiving the media query data including feature data of the query media, coordinate data, and boundary data, matching the features with corresponding features of an media database using the feature data to identify features in the media database within a predetermined hamming distance in a hash table from the corresponding features of the query media to obtain matched features in the media database, determining candidate media whose number of matched features exceeds a matched feature threshold, generating a geometry similarity score between the query media and each candidate media using the feature data and the coordinate data, generating a boundary similarity score between the query media and each candidate media using the boundary data, ranking the candidate media based on the numbers of matched features, the geometry similarity scores and the boundary similarity scores. 1. A method for determining one or more ranked candidate media in response to media query data generated at a mobile client device corresponding to a query media , comprising:receiving the media query data from the mobile client device, the media query data comprising feature data of one or more features of the query media encoded by one or more similarity preserving hashing functions and coordinate data for each of the one or more features;matching, on a server computer system, the one or more of the features of the query media with one or more corresponding features of a media database using the feature data of the query media to identify one or more features of the media database within a predetermined hamming distance in one or more hash tables from the one or more corresponding features of the query media to obtain one or more matched features in the media database;determining, on the server computer system, one or more candidate media whose number of matched features exceeds a matched ...

Rapid image annotation via brain state decoding and visual pattern mining

Human visual perception is able to recognize a wide range of targets but has limited throughput. Machine vision can process images at a high speed but suffers from inadequate recognition accuracy of general target classes. Systems and methods are provided that combine the strengths of both systems and improve upon existing multimedia processing systems and methods to provide enhanced multimedia labeling, categorization, searching, and navigation.

MULTIMEDIA INTEGRATION DESCRIPTION SCHEME, METHOD AND SYSTEM FOR MPEG-7

The invention provides a system and method for integrating multimedia descriptions in a way that allows humans, software components or devices to easily identify, represent, manage, retrieve, and categorize the multimedia content. In this manner, a user who may be interested in locating a specific piece of multimedia content from a database, Internet, or broadcast media, for example, may search for and find the multimedia content. In this regard, the invention provides a system and method that receives multimedia content and separates the multimedia content into separate components which are assigned to multimedia categories, such as image, video, audio, synthetic and text. Within each of the multimedia categories, the multimedia content is classified and descriptions of the multimedia content are generated. The descriptions are then formatted, integrated, using a multimedia integration description scheme, and the multimedia integration description is generated for the multimedia content. The multimedia description is then stored into a database. As a result, a user may query a search engine which then retrieves the multimedia content from the database whose integration description matches the query criteria specified by the user. The search engine can then provide the user a useful search result based on the multimedia integration description. 1. A method comprising:generating, for an identified multimedia type of multimedia content, multimedia object descriptions from multimedia objects in the multimedia content;generating an integration description scheme which creates, when implemented, relationships between the multimedia object descriptions and non-hierarchical entity relation graph descriptions, wherein the non-hierarchical entity relation graph descriptions relate to communication between the multimedia objects in the multimedia content;generating a description record to represent a portion of the multimedia content by integrating, according to the ...

METHODS AND SYSTEMS FOR AGGREGATION AND ORGANIZATION OF MULTIMEDIA DATA ACQUIRED FROM A PLURALITY OF SOURCES

A system for aggregation and organization of multimedia data acquired from a plurality of sources can include a data collection element, a multimedia segmentation element, a memory, an extraction element, and a user interface. The data collection element can be configured to receive multimedia data. The multimedia segmentation element can be coupled to the data collection element and can be configured to separate the multimedia data into a plurality of multimedia data segments. The memory can be configured to store hierarchical category information having a plurality of categories. The extraction element can be coupled to the multimedia segmentation element and the memory and can be configured to associate at least one of the plurality of categories with each of the plurality of multimedia data segments. 1. A system for aggregation and organization of multimedia data acquired from a plurality of sources , comprising:a data collection element configured to receive multimedia data;a multimedia segmentation element, coupled to the data collection element, and configured to separate the multimedia data into a plurality of multimedia data segments;a memory configured to store hierarchical category information having a plurality of categories;an extraction element, coupled to the multimedia segmentation element and the memory, configured to associate at least one of the categories with each of the multimedia data segments; anda user interface, coupled to the extraction element, configured to provide a user with at least one of the multimedia data segments associated with a corresponding one of the categories in the hierarchical category information.2. The system of claim 1 , wherein the data collection element further comprises a recorder configured to record audiovisual data.3. The system of claim 2 , wherein the recorder is configured to record at least one of analog video content and digital video content.4. The system of claim 2 , wherein the recorder further ...

SYSTEMS AND METHODS FOR AUTOMATICALLY DETERMINING AN IMPROVED VIEW FOR A VISUAL QUERY IN A MOBILE SEARCH

Systems and methods for automatically determining an improved view for a visual query in a mobile location or object search are provided. In some embodiments, methods for automatically determining an improved view for a visual query in a mobile location or object search system include obtaining at least one data set based on a prior visual query, wherein the at least one data set includes at least a top location or object and one or more other locations or objects; retrieving at least one distinctiveness measurement for one or more locations or objects in the at least one data set; and determining the improved view based on the retrieved at least one distinctiveness measurement. 1. A method for automatically determining an improved view for a visual query in a mobile search system , comprising:obtaining at least one result data set based on a prior visual query, wherein the at least one result data set includes at least a top result and one or more other results;retrieving at least one distinctiveness measurement for one or more views of one or more objects in the at least one result data set; anddetermining the improved view based on the retrieved at least one distinctiveness measurement.2. The method of claim 1 , wherein the determining the improved view comprises an information gain maximization process.3. The method of claim 1 , wherein the determining the improved view comprises a majority voting process.4. The method of further comprising estimating a viewing angle of the prior query and determining outlier results claim 1 , if any claim 1 , from the at least one result data set based on the estimated viewing angle of the prior query claim 1 , and removing from the at least one result data set the determined outlier results.5. The method of wherein the estimating the viewing angle of the prior query comprises using machine learning classification or local feature matching based image alignment.6. The method of claim 5 , wherein the machine learning ...

SYSTEM AND METHOD FOR MOTION ESTIMATION AND MODE DECISION FOR LOW-COMPLEXITY H.264 DECODER

The present invention relates to systems and methods for motion estimation and mode decision for low-complexity H.264 standard decoders. The present invention includes a method for optimizing the selection of motion vectors and motion compensation block modules in a video encoder in order to decrease the complexity of the video upon decoding. The novel method of the present invention may include novel steps for selecting motion vectors, block modes, and for applying a complexity-control algorithm to encode the received input video data sequence in accordance with the identified target complexity level. The present invention may be implemented in accordance with current and future video decoding standards to optimize decoding by reducing decoding complexity and thereby reducing required resources and power consumption. 1. A method for optimizing the selection of motion vectors and motion compensation block modules in a video encoder comprising:receiving an input video data sequence comprising at least one macroblock;identifying a target complexity level for the video data sequence;determining a Lagrange multiplier for the video data sequence;for each at least one macroblock, calculating at least one motion vector for one or more block modes based on the determined Lagrange multiplier;for each at least one macroblock, selecting one of the one or more block modes based on the determined Lagrange multiplier;encoding the received input video data sequence in accordance with the identified target complexity level to produce an encoded bitstream that may be decoded with consistent complexity.2. The method of claim 1 , wherein the step of encoding the received input video data sequence comprises applying a complexity control algorithm.3. The method of claim 1 , wherein the received input video data is data in accordance with the H.264 standard.4. The method of claim 1 , wherein the received input video data is data in accordance with the MPEG-4 standard.5. The method of ...

Video description system and method

Systems and methods for describing video content establish video description records which include an object set (24), an object hierarchy (26) and entity relation graphs (28). Video objects can include global ojects, segment objects and local objects. The video objects are further defined by a number of features organized in classes, which in turn are further defined by a number of feature descriptors (36, 38 and 40). The relationships (44) between and among the objects in the object set (24) are defined by the object hierarchy (26) and entity relation graphs (28). The video description records provide a standard vehicle for describing the content and context of video information for subsequent access and processing by computer applications such as search engines, filters, and archive systems.

Extraction of high-level features from low-level features of multimedia content

A method extracts high-level features from a video including a sequence of frames. Low-level features are extracted from each frame of the video. Each frame of the video is labeled according to the extracted low-level features to generate sequences of labels. Each sequence of labels is associated with one of the extracted low-level feature. The sequences of labels are analyzed using learning machine learning techniques to extract high-level features of the video.

Method And System For Optimal Video Transcoding Based On Utility Function Descriptors

Techniques for generating utility-based descriptors from compressed multimedia information are disclosed. A preferred method includes the steps of receiving least a segment of compressed multimedia information, determining two or more portions of utility based descriptor information based on one or more adaptation operations, each corresponding to a unique target rate, adapting the compressed multimedia segment by each the portions of utility based descriptor information to generate adapted multimedia segments, using a quality management method to generate measurement for each adapted multimedia segment, and generating a utility based descriptors based on the portions of utility based descriptor information and corresponding quality measurements.

System and method for dynamically and interactively searching media data

The disclosed subject matter is directed to systems and methods for searching a database of media content wherein the user can dynamically and interactively perform searches and navigate search results. One or more search anchors are received, and at least one of the search anchors is associated with an anchor cell on a navigation map. One or more documents assigned to at least one cell on the navigation map can be determined, and the cells are populated with search results based at least in part on the search anchors. At least one of the documents is then displayed to a user.

Image description system and method

Systems and methods for describing image content establish image description records which include an object set (24), an object hierarchy (26) and entity relation graphs (28). For image content, image objects can include global objects (O0 8) and local objects (O1 2 and O2 6). The image objects are further defined by a number of features of different classes (36, 38 and 40), which in turn are further defined by a number of feature descriptors. The relationships between and among the objects in the object set are defined by the object hierarchy (26) and entity relation graphs (28). The image description records provide a standard vehicle for describing the content and context of image information for subsequent access and processing by computer applications such as search engines, filters, and archive systems.

Methods and systems for generating multimedia signature

Techniques for signing multimedia content and verifying received multimedia content that are both robust and accurate are provided. Invariant features are extracted from multimedia content, and certain attributes are computed. Multimedia content is quantized, and extracted invariant features and quantized original multimedia content are encrypted to form a digital signature. The multimedia content and digital signature may be verified even after the introduction of distortions by using content registration. In a preferred embodiment, a refined authentication technique is used to obtain a continuous distance measure, to verify the authenticity of multimedia content based on a pre-defined threshold.

Method and system for generating semantic visual templates for image and video retrieval

For database image/video retrieval, a semantic visual template (SVT) is a set of icons of example scenes/objects that characterize a concept, e.g. skiing, sunset and the like. SVTs provide for two-way interaction between a user and a system. The user can provide the system with an initial sketch or example image, as a seed to the system to automatically generate other representations of the same concept. The user then can pick those views for inclusion that are plausible for representing the concept. When an SVT has been established, the database can be searched with it, for the user to provide relevancy feedback on the returned results. With established SVTs, the user can interact with the system at concept level. In forming new concepts, pre-existing SVTs can be used. A limited vocabulary can be parsed, in conjunction with semantic visual templates for querying the system.

Multimedia integration description scheme, method and system for mpeg-7

The invention provides a system and method for integrating multimedia descriptions in a way that allows humans, software components or devices to easily identify, represent, manage, retrieve, and categorize the multimedia content. In this manner, a user who may be interested in locating a specific piece of multimedia content from a database, Internet, or broadcast media, for example, may search for and find the multimedia content. In this regard, the invention provides a system and method that receives multimedia content and separates the multimedia content into separate components which are assigned to multimedia categories, such as image, video, audio, synthetic and text. Within each of the multimedia categories, the multimedia content is classified and descriptions of the multimedia content are generated. The descriptions are then formatted, integrated, using a multimedia integration description scheme, and the multimedia integration description is generated for the multimedia content. The multimedia description is then stored into a database. As a result, a user may query a search engine which then retrieves the multimedia content from the database whose integration description matches the query criteria specified by the user. The search engine can then provide the user a useful search result based on the multimedia integration description.

Video concept classification using audio-visual atoms

A method for determining a classification for a video segment, comprising the steps of: breaking the video segment into a plurality of short-term video slices, each including a plurality of video frames and an audio signal; analyzing the video frames for each short-term video slice to form a plurality of region tracks; analyzing each region track to form a visual feature vector and a motion feature vector; analyzing the audio signal for each short-term video slice to determine an audio feature vector; forming a plurality of short-term audio-visual atoms for each short-term video slice by combining the visual feature vector and the motion feature vector for a particular region track with the corresponding audio feature vector; and using a classifier to determine a classification for the video segment responsive to the short-term audio-visual atoms.

Method and system for generating semantic visual templates for image and video retrieval

For database image/video retrieval, a semantic visual template (SVT) is a set of icons of example scenes/objects that characterize a concept, e.g. skiing, sunset and the like. SVTs provide for two-way interaction between a user and a system. The user can provide the system with an initial sketch or example image, as a seed to the system to automatically generate other representations of the same concept. The user then can pick those views for inclusion that are plausible for representing the concept. When an SVT has been established, the database can be searched with it, for the user to provide relevancy feedback on the returned results. With established SVTs, the user can interact with the system at concept level. In forming new concepts, pre-existing SVTs can be used. A limited vocabulary can be parsed, in conjunction with semantic visual templates for querying the system.

Fundamental entity-relationship models for the generic audio visual data signal description

Systems and methods for generating standard description records from multimedia information are provided. The invention utilizes fundamental entity-relation models for the Generic AV DS that classify the entities, the entity attributes, and the relationships in relevant types describing visual data. It involves classification of entity attributes into syntactic and semantic attributes. Syntactic attributes can be categorized into different levels: type/technique, global distribution, local structure, and global composition. Semantic attributes can be categorized into different levels: generic object, generic scene, specific object, specific scene, abstract object, and abstract scene. The invention further utilizes classification of entity relationships into syntactic and semantic categories. Syntactic relationships can be categorized into spatial, temporal, and visual categories. Semantic relationships can be categorized into lexical and predicative categories.

Watermarking of digital image data

Digital watermarks can serve to indicate copyright ownership of digitized video. When video images are transmitted as transformed by discrete cosine transformation (DCT) for compression, with or without motion compensation, it is advantageous to include a watermark after transformation. To this end, a DCT watermark is generated for optimal visibility based on the original image data, and the generated watermark is superposed on the transformed data.

Method and apparatus for processing echocardiogram video images

Methods and a system are disclosed for processing an echocardiogram video of a patient's heart. The echocardiogram comprises at least a first sequence of consecutive video frames corresponding to a first view of the patient's heart concatenated with a second sequence of consecutive video frames corresponding to a second view of the patient's heart. The end-diastole phase of the patient's heart is monitored in each frame by detecting the electrocardiograph wave, and a key frame is selected upon the occurence of the R-wave peak in the electrocardiograph wave in each of the first sequence of consecutive video frames and in the second sequence of consecutive video frames. The shape and color content of the echocardiogram image window is monitored in certain video frames, and a transition is detected when there is a change in the first feature between adjacent frames. A summary is generated which comprises by the video frames corresponding to the end-diastole phase.

Multimedia integration description scheme, method and system for mpeg-7

The invention provides a system and method for integrating multimedia descriptions in a way that allows humans, software components or devices to easily identify, represent, manage, retrieve, and categorize the multimedia content. In this manner, a user who may be interested in locating a specific piece of multimedia content from a database, Internet, or broadcast media, for example, may search for and find the multimedia content. In this regard, the invention provides a system and method that receives multimedia content and separates the multimedia content into separate components which are assigned to multimedia categories, such as image, video, audio, synthetic and text. Within each of the multimedia categories, the multimedia content is classified and descriptions of the multimedia content are generated. The descriptions are then formatted, integrated, using a multimedia integration description scheme, and the multimedia integration description is generated for the multimedia content. The multimedia description is then stored into a database. As a result, a user may query a search engine which then retrieves the multimedia content from the database whose integration description matches the query criteria specified by the user. The search engine can then provide the user a useful search result based on the multimedia integration description.

Method and system for optimal video transcoding based on utility function descriptors

Techniques for generating utility-based descriptors (210) from compressed multimedia information are provided. A preferred method includes the steps of receiving (230) at least a segment of compressed multimedia information, determining two or more portions of utility based descriptor information based on one or more adaptation operations, each corresponding to a unique target rate, adapting the compressed multimedia segment by each of the portions of utility based descriptor information to generate adapted multimedia segments, using a quality of management method to generate a quality measurement for each adapted multimedia segment, and generating a utility based descriptors based on the portions of utility based descriptor information and corresponding quality measurements.

System und verfahren zur bildbeschreibung

Videobeschreibungssystem und -verfahren

Method and system for indexing and content-based adaptive streaming of digital video content

Adaptable Automated Interpretation of Rapid Diagnostic Tests Using Self-Supervised Learning and Few-Shot Learning

A framework for a few-shot learning method is disclosed. In a first part, self-supervision and classification supervision are used to train a feature extractor. An example self-supervision method comprises running grayscale images through an edge filter, normalizing the filtered images, setting the normalized images to ground truth, generating feature-extracted images, using a decoder to reconstruct images from the feature-extracted images, determining a loss between the reconstructed images and the ground truth images, and using the loss to update parameters of the feature extractor. In a second part, a few-shot adaptation process is performed to adapt the model to a novel rapid test kit.

Systems and methods for interoperable multimedia content descriptions

Systems and methods for generating standard description records from multimedia information are provided. The system includes at least one multimedia information input interface (180) receiving multimedia information, a computer processor, and a data storage system (150), operatively coupled to said processor, for storing said at least one description record. The processor performs object extraction processing to generate multimedia object descriptions (200, 201, 205) from the multimedia information, and object hierarchy processing (410, 420) to generate multimedia object hierarchy descriptions, to generate at least one description record including the multimedia object descriptions (200, 201, 205) and multimedia object hierarchy descriptions for content embedded within the multimedia information.

Algorithms and system for object-oriented content-based video search

Object-oriented methods and systems for permitting a user to locate one or more video objects from one or more video clips over an interactive network are disclosed. The system includes one or more server computers (110) comprising storage (111) for video clips and databases of video object attributes, a communications network (120), and a client computer (130). The client computer contains a query interface to specify video object attribute information, including motion trajectory information (134), a browser interface to browse through stored video object attributes within the server computers, and an interactive video player.

Systems and methods for interoperable multimedia content descriptions

Systems and methods for generating standard description records from multimedia information are provided. The system includes at least one multimedia information input interface (180) receiving multimedia information, a computer processor, and a data storage system (150), operatively coupled to said processor, for storing said at least one description record. The processor performs object extraction processing to generate multimedia object descriptions (200, 201, 205) from the multimedia information, and object hierarchy processing (410, 420) to generate multimedia object hierarchy descriptions, to generate at least one description record including the multimedia object descriptions (200, 201, 205) and multimedia object hierarchy descriptions for content embedded within the multimedia information.

System and method for motion estimation and mode decision for low-complexity h.264 decoder

The present invention relates to systems and methods for motion estimation and mode decision for low-complexity H.264 standard decoders. The present invention includes a method for optimizing the selection of motion vectors and motion compensation block modules in a video encoder in order to decrease the complexity of the video upon decoding. The novel method of the present invention may include novel steps for selecting motion vectors, block modes, and for applying a complexity-control algorithm to encode the received input video data sequence in accordance with the identified target complexity level. The present invention may be implemented in accordance with current and future video decoding standards to optimize decoding by reducing decoding complexity and thereby reducing required resources and power consumption.

複数の状態に基づいて連続した圧縮映像を解析する方法

【課題】ドメイン固有の画像の構造解析を効率よく行う 【解決手段】連続する圧縮映像の高レベルシンタックスおよび構造を複数の状態に基づいて解析する方法である。まず、既知の状態に分割されたトレーニング映像を使用して、各状態の隠れマルコフモデルの集合をトレーニングする。次に、連続した圧縮映像の固定長スライディングウィンドウからドメイン固有の特徴の集合を抽出し、トレーニングした隠れマルコフモデルの集合を使用して各ドメイン固有の特徴の集合ごとに最大尤度の集合を決定する。最後に、ダイナミックプログラミングを最大尤度の各集合に適用して、圧縮映像フレームの各固定長スライディングウィンドウの特定の状態を決定する。 【選択図】　　　　図３

Method and electronic device for classifying a video content item

The method of the invention consists of comparing (19) a color representation of a video content item with typical color representation of a plurality of video content genres and determining (21) a video content genre from the plurality of video content genres based on a result of the comparison. The computer program product of the invention renders a programmable computer capable of performing the method of the invention. The electronic device of the invention is equipped with electronic circuitry which is capable of comparing a color representation of the video content item with typical color representations of a plurality of video content genres and determining a video content genre from the plurality of video content genres based on a result of the comparison.

Method and apparatus for watermarking and authenticating data

A method and apparatus for watermarking and authenticating data. Features codes are derived from multimedia data (e.g., image data) and used to derive a quasi-random watermark and an encrypted signature. In two stages of watermarking, the quasi-random watermark and the signature are embedded in the multimedia data to derive watermarked data. In an authentication procedure, the watermarked data are authenticated by newly deriving the feature codes, extracting the watermarks, and determining whether the extracted watermarks correspond to the newly derived feature codes. In addition, the embedded signature is extracted from the data and decrypted. The newly derived feature codes are used to derive data which is compared to the decrypted signature. The user of the above procedures can select a desired authentication strength based upon the percentage of the multimedia data used to derive the feature codes.

System and method for extracting text captions from video and generating video summaries

Method and system for watermarking an electrically depicted image

A system for watermarking (222) an image file selects coefficients using a selection procedure that is kept secret, and assigns the selected coefficients to coefficient pairs. To detect an unauthorized alteration after the image file has been watermarked, coefficient pairs are selected using the same secret procedure that was originally used to generate the raw signature (418) values. The raw signature values may be grouped into sets, which are mapped onto shortened signature (422) codes having reduced number of bits.