RFC: Consider cache configuration in new girder worker tasks

[zachmullen]

In the old G_W architecture, the girder_io plugin on the worker side could be configured at deploy time to cache files downloaded from Girder. If a user wanted to do that in the new architecture, they'd need to configure the GirderClient object on the transforms themselves to set up caching, meaning they'd have to set up the caching policies on the Girder server side that was calling the G_W task. This seems a bit counterintuitive to me, but I'm not sure of the best way around that issue. Ideally there could be some way to configure just girder_worker to do the caching without the server-side task calling code having to worry about it.

[kotfic]

Alternatively we could make the celery application object (or even the individual task object) available to the transform and it could pull configuration from those objects. E.g. by setting these directly on the transforms just before here and here

[kotfic]

Something like

def _maybe_transform_argument(self, arg, **kwargs):
        if hasattr(arg, 'transform') and six.callable(arg.transform):
            arg.app, arg.task = self.app, self
            return arg.transform(**kwargs)
        return arg

I don't know if this is actually a good idea or not. Its kind of elegant that the transforms as they exist are agnostic towards being used in the context of celery. On the other hand houses are meant to be lived in, not admired. 🤷‍♂️