Semantic video search by automatic video annotation using TensorFlow | Semantic Scholar (2024)

Distinctive methodologies of automatic video annotation are discussed, which are classified into five classes namely, Generative models, Distance-based similarity model, Discriminative model, Ontology-based models, Deep Learning- based models, and Deep Learning -based models.

This survey paper briefly reviews novel representative literature and systematizes the commonalities in different machine learning techniques such as supervised learning, active learning, transfer learning, and neural networks and summarized the performances of all the mentioned approaches applied to various datasets.

An approach that allows the systematic video analysis using deep learning and ontology generation and permits the extraction and the building of an ontology using results obtained by the deep learning techniques such as key frames, detected objects and actions.

An approach is presented that allows the systematic video analysis using deep learning and ontology generation and permits the extraction and the building of an ontology using results obtained by the deep learning techniques such as key frames, detected objects and actions.

This work proposes a novel framework called Video Annotator (VA), which leverages the zero-shot capabilities of vision-language foundation models combined with active learning techniques, and demonstrates that VA enables the efficient creation of high-quality models.

This survey paper provides a review of previous techniques that focuses on optimized data annotation and labeling for video, audio, and text data and suggests ways to reduce the data preparation cost.

The proposed approach splits video in set of images, detects objects/entities present in these images and stores their descriptions into a standard XML file and is able to track motion that animated entities in the video sequences.

This article provides a Deep Learning algorithm using TensorFlow for tracking motion and animated entities in video sequences and uses images from video splitting to detect objects / entities present and store their descriptions in a standard XML file.

A No-code MLOps-based data annotation system that empowers non-developers to train and apply auto-labeling models seamlessly, enabling versatile applications and marking a step towards democratizing data annotation.

This research aims to develop automatic ways to generate narratives by combining short video segments and tailoring this to the learner’s needs by proposing an original framework to characterise video segments for learning by combining video content and audience attention.

...

...

Methods for video structure analysis, including shot boundary detection, key frame extraction and scene segmentation, extraction of features including static key frame features, object features and motion features, video data mining, video annotation, and video retrieval including query interfaces are analyzed.

It is demonstrated that precision-recall can be significantly improved by taking ontology into account and this learning algorithm can be used for multiple topic document classification such as Internet documents and video documents.

PYVIDID is a Python based video identification system able to match and search query videos to a large database and clearly shows that perceptual hash algorithms can indeed be used for video identification with high accuracy.

Experimental results show the proposed algorithm for extracting invariant salient features from images and video sequences is very effective for recognizing objects in images/video with viewpoint changes and background variations.

Modelling deep neural networks with rectified linear unit (ReLU) non-linearities with minimal human hyper-parameter tuning on a 50-hour English Broadcast News task shows an 4.2% relative improvement over a DNN trained with sigmoid units, and a 14.4% relative improved over a strong GMM/HMM system.

The TensorFlow interface and an implementation of that interface that is built at Google are described, which has been used for conducting research and for deploying machine learning systems into production across more than a dozen areas of computer science and other fields.

This paper examines the results of applying Term Frequency Inverse Document Frequency to determine what words in a corpus of documents might be more favorable to use in a query and provides evidence that this simple algorithm efficiently categorizes relevant words that can enhance query retrieval.

It is apparent from the reviews that current research into semi-automatic or automatic ontology research in all the three aspects of generation, mapping and evolving have so far achieved limited success.

...

...