Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow `docker push' to push multiple/a subset of tags #267

Open
ebd2 opened this issue Jun 29, 2017 · 61 comments
Open

Allow `docker push' to push multiple/a subset of tags #267

ebd2 opened this issue Jun 29, 2017 · 61 comments

Comments

@ebd2
Copy link

ebd2 commented Jun 29, 2017

Allow the following usage:

docker push org/image-a:some_tag org/image-b:other_tag

This would make docker push simpler to use in systems that need to build and push multiple images to dockerhub by obviating the need to wrap docker push in a loop.

@glyif
Copy link

glyif commented Aug 11, 2017

@dnephin, super cool tool! imma try it out. for my educational purposes, i'm going to implement this feature. :)

@iBobik
Copy link

iBobik commented Nov 15, 2017

Also it can save resources while building - the same image does not have to be uploaded twice.

@thaJeztah
Copy link
Member

Prior discussions on this feature can be found in moby/moby#7336, moby/moby#11682, moby/moby#16106, and related, moby/moby#23383

Note that adding support for this on the command-line will not provide enhancements other than not having to "loop" through tags for pushing (see #458 (review))

@SeidChr
Copy link

SeidChr commented Nov 25, 2017

I wonder how that ticket is still open. it started in 2014.. was a quest to finally find the last instance here :)
Anyway. Im in for it.

@elboletaire
Copy link

elboletaire commented Dec 18, 2017

I wonder how that ticket is still open. it started in 2014.. was a quest to finally find the last instance here :) Anyway. Im in for it.

+1

It's curious/weird how you can tag multiple tags at once with docker build -t, but you can't publish them all at once...

@Hronom
Copy link

Hronom commented Dec 22, 2017

Guys what the progress on this issue? Still need docker push with multiple tags!

@Nowaker
Copy link

Nowaker commented Jan 17, 2018

This, please!

@smuryginim
Copy link

+1

1 similar comment
@XBeg9
Copy link

XBeg9 commented Apr 5, 2018

+1

@pappnase99
Copy link

How is this still a thing?

@elboletaire
Copy link

elboletaire commented Apr 14, 2018

There's currently the PR #458 (review) which is under review. AFAIK for now it only adds a loop for the docker push instead of trying to do it in parallel, which is how it works in docker build; that's why it isn't still accepted.

@ggalihpp
Copy link

ggalihpp commented Jul 6, 2018

Oh its open. I though its already implemented :(

@AntonKrug
Copy link

Ignoring the performance, the options either to push image with all tags or single tag seems bit odd.

@javatechy
Copy link

You can install dokr . and use dokr dock --push image_name. This will push your multiple images

@Hronom
Copy link

Hronom commented Sep 26, 2018

Please do this at least, so many time since this started, I was thinking it's already implemented.

@aramhakobyan
Copy link

+1

@achillesrasquinha
Copy link

achillesrasquinha commented Dec 7, 2018

This is more of a design problem. I'd blindly assumed that this would have worked for me

$ docker push $(docker images --format "{{.Repository}}" | grep "foobar")

only to realize it takes just one argument. I'd say the same for docker pull. There are just too many great use cases for a multi-push. Plus, it's pretty neat instead of a

$ for i in `docker images --format "{{.Repository}}" | grep "foobar"; do docker push $i; done

Not very command-like.

@lagden
Copy link

lagden commented Jan 10, 2019

@achillesrasquinha

Is it not necessary pass the tag?

Example:

docker images --format "{{.Repository}}:{{.Tag}}" | grep "foobar"
lagden/foobar:6.1.0
lagden/foobar:latest

@purplesoda
Copy link

purplesoda commented Jan 14, 2019

I was able to do this using AWS ECR for our docker image hosting. We're using jenkins for CI to push the image up to ECR once the build is finished. Then you can use aws cli to re-tag the image with whatever you want, including variables.

In this example, push your initial image up with the latest tag, then specify whatever additional tag you'd like in the second command behind --image-tag

MANIFEST=\$(aws ecr batch-get-image --repository-name yeet --image-ids imageTag=latest --query 'images[].imageManifest' --output text)

aws ecr put-image --repository-name yeet --image-tag v_$BUILD_NUMBER --image-manifest "\$MANIFEST"

We're using ECS for hosting so this actually fit nicely into our pipe. You can still use amazon ECR if you're hosting elsewhere though.

@thiagolsfortunato
Copy link

Any new?!

@davidtabernerom
Copy link

I achieve this by using pipes with echo and xargs, but it really needs to be implemented, this only automate it, but the command is summoned each time.

If anyone wondering:
echo 'repo/flavor:tag' 'repo/flavor:tag' 'repo/flavor:tag' | xargs -n 1 docker push

@saravana1992
Copy link

Any news on this?

@sj26
Copy link
Contributor

sj26 commented Sep 15, 2019

Abandon hope all ye who enter here.

I've had great success using skopeo which allows flexible manipulation of remote docker image registries, like re-tagging images on remotes without needing to push/pull locally, or pushing/pulling multiple images in parallel, etc.

@hasokeric
Copy link

@solvaholic: Sorry I missed your comment of many months ago. I no longer build software; I now make furniture out of wood. The hours are long, the pay sucks, and there's always the opportunity to remove my finger with a table saw, but nobody asks me if I can add an RSS feed to a DBMS, so there's that :-)

@ebd2 to clarify your request, and to help future readers find the option to push all tags, would you please update the title and issue body to show you'd like to push a subset of tags for an image repository? For example:

Allow `docker push' to push multiple a subset of tags

I've edited as "multiple/a subset of" for clarity. If you think this needs further clarification, try @ mentioning me again; I caught this comment because I was cleaning out the email account github emails go to. Now that the signal to noise ratio is hopefully higher, I might catch messages again!

He will be back! Soon.

@seanmcoleman
Copy link

@solvaholic: Sorry I missed your comment of many months ago. I no longer build software; I now make furniture out of wood. The hours are long, the pay sucks, and there's always the opportunity to remove my finger with a table saw, but nobody asks me if I can add an RSS feed to a DBMS, so there's that :-)

@ebd2 to clarify your request, and to help future readers find the option to push all tags, would you please update the title and issue body to show you'd like to push a subset of tags for an image repository? For example:

Allow `docker push' to push multiple a subset of tags

I've edited as "multiple/a subset of" for clarity. If you think this needs further clarification, try @ mentioning me again; I caught this comment because I was cleaning out the email account github emails go to. Now that the signal to noise ratio is hopefully higher, I might catch messages again!

May I recommend the SawStop?

@htoooth
Copy link

htoooth commented Sep 22, 2020

sound good

@warent
Copy link

warent commented Sep 22, 2020

@docyx Cool your jets. If you disagree with a comment, you can also use a 👎 or 😕 reaction instead of filling up feeds with off-topic verbal abuse. @AlphaQx did not deserve that.

@navilan
Copy link

navilan commented Sep 22, 2020

@solvaholic: Sorry I missed your comment of many months ago. I no longer build software; I now make furniture out of wood. The hours are long, the pay sucks, and there's always the opportunity to remove my finger with a table saw, but nobody asks me if I can add an RSS feed to a DBMS, so there's that :-)

@ebd2 to clarify your request, and to help future readers find the option to push all tags, would you please update the title and issue body to show you'd like to push a subset of tags for an image repository? For example:

Allow `docker push' to push multiple a subset of tags

I've edited as "multiple/a subset of" for clarity. If you think this needs further clarification, try @ mentioning me again; I caught this comment because I was cleaning out the email account github emails go to. Now that the signal to noise ratio is hopefully higher, I might catch messages again!

This is really funny comment

If it's a funny comment, use the 😄 reaction, don't fill up my fucking feed with this bullshit

Fuck you

This is a really funny comment :)

@Oymate
Copy link

Oymate commented Sep 22, 2020

@ devs limit discussion

@Fasani
Copy link

Fasani commented Sep 22, 2020

@solvaholic do you at least sell it online? 😂

@maxsatula
Copy link

@solvaholic: Sorry I missed your comment of many months ago. I no longer build software; I now make furniture out of wood. The hours are long, the pay sucks, and there's always the opportunity to remove my finger with a table saw, but nobody asks me if I can add an RSS feed to a DBMS, so there's that :-)

He will be back! Soon.

If that really happens, hope he wasn't an Emacs user. As it strictly requires all the 10 fingers (sometimes even more), you know...

@ebd2
Copy link
Author

ebd2 commented Oct 2, 2020

If that really happens, hope he wasn't an Emacs user. As it strictly requires all the 10 fingers (sometimes even more), you know...

At the risk of taking us even further off topic (and further vexing the Docker maintainers), I broke my right wrist almost 20 years ago, and I find Emacs outrageously uncomfortable to use with a nomal-ish keyboard layout (QWERTY with a couple of remaps to make it closer to the old-school Sun layout). As I don't have this issue with vi/vim, that's my preferred editor.

And yes, I still futz around with code from time to time, just not for money. It's a lot more fun that way.

@osher
Copy link

osher commented Oct 8, 2020

@cpuguy83 - I'm not afraid of loops. This is not my problem.

The use-case is to avoid instability where some tags have been published and some have not.
Looping makes a separate API call per tag, and the danger is a network hickup in the middle of this loop and a careless CI developer somewhere out there.

What I expect from such API is to apply ALL tags provided and exit well, or NONE of them and exit with an error.

Use-Case:
Consider a docker-based containerized micro-services mesh.
Consider a CI of a mono-repo that builds it all and rolls out all applicable changes
Consider a strategy that relays on tag latest.

  • sounds familiar, or not common enough? even without the latest tag it's still a thing.

If following a build only a part of the images that should have been tagged latest have in deed accepted the tag - we're in an unstable state which I hoped to avoid.

The solution now is a whole bunch of scriptology that handles many concerns and more, where the true concerns of my code as a user are naming the images and sending success/fail notifications from the CI.

Perhaps this requires evolution also on the registry side - I'm not involved enough to know.
But if we mean to provide a good service to our users - this should be the mindset.

@antonu17
Copy link

antonu17 commented Nov 5, 2020

but nobody asks me if I can add an RSS feed to a DBMS, so there's that :-)

The best description for a Debezium Kafka Connector ever

@ferrywlto
Copy link

@cpuguy83 - I'm not afraid of loops. This is not my problem.

The use-case is to avoid instability where some tags have been published and some have not.
Looping makes a separate API call per tag, and the danger is a network hickup in the middle of this loop and a careless CI developer somewhere out there.

What I expect from such API is to apply ALL tags provided and exit well, or NONE of them and exit with an error.

Use-Case:
Consider a docker-based containerized micro-services mesh.
Consider a CI of a mono-repo that builds it all and rolls out all applicable changes
Consider a strategy that relays on tag latest.

* sounds familiar, or not common enough? even without the `latest` tag it's still a thing.

If following a build only a part of the images that should have been tagged latest have in deed accepted the tag - we're in an unstable state which I hoped to avoid.

The solution now is a whole bunch of scriptology that handles many concerns and more, where the true concerns of my code as a user are naming the images and sending success/fail notifications from the CI.

Perhaps this requires evolution also on the registry side - I'm not involved enough to know.
But if we mean to provide a good service to our users - this should be the mindset.

In addition, executing the command multiple times for the same image with different tags could be a problem to some CI/CD pipeline that cost network traffic.

@cpuguy83
Copy link
Collaborator

@osher Atomic push of multiple tags is indeed something that would need to live within the registry specification. This sounds like a different case for sure.

@ferrywlto

In addition, executing the command multiple times for the same image with different tags could be a problem to some CI/CD pipeline that cost network traffic.

I'm not sure I understand this.

@DocX
Copy link

DocX commented Jun 10, 2021

In addition, executing the command multiple times for the same image with different tags could be a problem to some CI/CD pipeline that cost network traffic

Pushing same image multiple times (even with different tags), results in almost no traffic, since the registry will already have the same layers, and it will just skip them.

@Nowaker
Copy link

Nowaker commented Jun 10, 2021

Requests are still made. And CPU time is still lost. A push that shouldn't technically transfer anything takes 10 seconds in our CI. Multiply by 5 images like that and it's a minute of CI time lost on Docker nonsense.

TOP1 most upvoted feature request. This is where the discussion ends, and no technical reason for not implementing this feature is valid.

@cpuguy83
Copy link
Collaborator

@Nowaker Requests to the docker CLI are cheap. The requests to the registry are expensive. Having docker support multiple tags on push does not change the number of requests that have to be made to the registry.
For sure, round trips to DockerHub are expensive but there is no amount of changing Docker that will fix that.

@cpuguy83
Copy link
Collaborator

There may be an exception here for auth tokens. In which case I'd recommend pushing with buildkit instead of docker push because that does some auth token re-use as I recall.

@fmichaut-diff
Copy link

fmichaut-diff commented Sep 20, 2023

I think the issue with the current solutions proposed to that problem, is that they would allow to push multiple images at once, which would requires multiple HTTP requests, which is no better than simply calling docker push twice.
(Or would requires changes in docker backend / registry for supporting a better way)

But the issue people are usually facing is not to push multiple images at once, but push multiple tags of the same image at once. Yes, there is already the --all-tags option which does so in 1 request (- correct me if i'm wrong here), but sometimes we want to push only a subset of tags.
(For instance: when testing locally, I can have a bunch of test images on my machine, but when I deploy, I only want to push the last image with tags latest, vX.Y.Z and vX-latest)

It is not very practical to call docker push multiple times, for the same image 3 times. We know the image is already uploaded, but docker push has to check every layer each time, just to upload the tag (which is like 10 chars max).

Since docker cli already has the --all-tags option which allows to send a list of tags alongside an image, could we implement something like the following :

docker push --tags 'tag1,tag2,latest' org/image

Or

docker push org/image:tag1,tag2,latest

I am assuming the registry already supports sending multiple tags with the image (since that's what the --all-tags option seems to do), so the only work required would be in the CLI.

@grantsunny
Copy link

This issue is still here? Just leave a name and looking forward for any updates.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests