Description
This issue was raised in developers mailing list. I think this is an interesting scope we should explore from tensorflow-io.
The original paper referred is:
FaceNet: A Unified Embedding for Face Recognition and Clustering
See below from the email:
I am investigating best practices in Tensorflow 2.0 for creating dynamic dataset pipelines for batch mining, but as far as I can see all common practices seem to only support fixed pipelines, typically using TFRecords.
Whereas this may work very well for e.g. training classifiers, it is common practice for face recognition and person re-identification to use more dynamic handling of the input pipeline.
Triplet training is a classic example that is first described in a paper from Google from 2015 (but there are also other schemes that use dynamic batch-mining):
https://arxiv.org/abs/1503.03832
There are at least 3 things here that I don't see can be supported with the current "fixed-pipeline" approaches available in Tensorflow:
1: Typically when doing triplet training you want each training batch to contain triplets of images that generate a moderate loss. This means before you know if you want to use any given triplet for the current training batch, you want to run an inference pass on it to check if it is a valid triplet.
2: Also you do NOT want to format the dataset as a series of pre-packaged triplets since for e.g. a dataset of ~10000 images with say 10 images per person identity, you will get someting like 15 billion possible triplet combinations. Packaging this into TFRecord with triplets is impractical, if not impossible, to do. Therefore you want the dataset pipeline to have access to inference passes on the CNN with the current weights when it generates triplets, and you want to have the possibility to select randomly from the entire set of identities.
3: Also, it makes sense to have simultaneous training from several datasets and device a batch-mining scheme that accesses multiple datasets in a customizable way.
Is there any recommended way of achieving this, or is this possibly a use-case that we would need to develop support for?