Add support --privileged when create service

[jimmyxian]

When create some container, we need privileged=True. But swarmkit does not support this.
I want to implement this by the following steps:

• Add privileged field in ContainerSpec.
• Pass privileged to HostConfig when create container.

If it's the right direction, I will submit a PR. :)

Also, there so many container options in docker-engine. If all the options are supported in that way, It's a huge work.Maybe we need have a good way to support all the options?

[dmcgowan]

Privileged containers are needed for any use case which needs to spin up Docker-in-Docker, a common pattern for testing not only docker itself but complex build setups such as allowed by https://github.com/docker/golem. I am looking forward to porting golem to swarm when this is enabled.

[stevvooe]

Also, there so many container options in docker-engine. If all the options are supported in that way, It's a huge work.Maybe we need have a good way to support all the options?

We don't actually want to do this. There are number of things that are hard or impossible to support in a clustered environment. There are other things that are downright insecure, such as privileged, that need to be rethought.

What aspect of privileged containers are you using? For example, what part enables DIND to work?

[dmcgowan]

We don't actually want to do this.

Many people will and it is a common pattern. However I do agree that time should be spent to figure out how to do it right since adding this will mean it can never be removed along with any security headaches it brings.

As for DIND in particular. I am not sure of all the settings that would need to be added to enable this, getting this working I think would cover many use cases for CI. Try running the make test run command without the privileged flag will yield...

mount: permission denied
Could not mount /sys/kernel/security.
AppArmor detection and --privileged mode might break.
mount: permission denied

The reason is obvious from https://github.com/docker/docker/blob/master/hack/dind.

[stevvooe]

@dmcgowan Supporting all of the container and host config options levied a massive amount of complexity on the swarm project for little gain. Each feature we add in services needs to be well-considered and designed for operation in a clustered environment. This is especially true for anything that requires privileged execution. (interesting aspect: ContainerSpec ends up looking a lot like HostConfig container.Config, in an image).

Right now, swarm services aren't really setup well for CI or build use cases. We don't really have the concept of batch jobs and collecting logs is problematic. This will come in the future.

Do we have any online use cases for actual services?

[dmcgowan]

Looking at the container setup code within docker the following is done by setting the privileged flag

• Set empty process label
• Warn of incompatibility with user namespaces
• Add all host devices from /dev
• Add device cgroup access rwm allow, see https://github.com/docker/docker/blob/master/daemon/oci_linux.go#L92
• Add all capabilities
• Clear read only flag for /sys mount
• Set read only paths to nil (*specs.Spec).Linux.ReadonlyPaths = nil, see https://github.com/docker/docker/blob/master/daemon/oci_linux.go#L561
• Set masked paths to nil (*specs.Spec).Linux.MaskedPaths = nil, see https://github.com/docker/docker/blob/master/daemon/oci_linux.go#L562
• Clear read only flag for cgroup mount
• Set app armor profile "unconfined"

[stevvooe]

@dmcgowan Wow, thanks for the overview!

cc @aluzzardi

[chanwit]

Global service fits really well for deploying monitoring agents on every node. And most of those agents require privileged access to some /proc. In my case, I want to monitor conntrack.

Please add support for this privileged flag, cc @aluzzardi

[stevvooe]

@chanwit: when not running with --privileged, what error do you get when trying to monitor conntrack?

[aluzzardi]

@stevvooe @chanwit @jimmyxian I was initially against adding --privileged. Seeing how many requests we got for this, I'm in favor for adding it now

[jimmyxian]

@aluzzardi Thanks, I will rebase the PR. :)

[chanwit]

@stevvooe my bad. it's another issue :-(
not relating to privileged.

Anyway I gave up running monitoring agents inside container for now.

[sashkachan]

Trying to run weave-scope using the new swarm, also hitting the roadblock with the privileged mode (or the lack thereof). Would be great to have it supported.