Download artifacts from Jenkins using Shell script

Download artifacts from Jenkins using Shell script

Recently, I had to download files from a Jenkins server using shell scripts. Since Jenkins offers permanent urls to the latest stable artifacts along with a nice XML/JSON API, this can be done pretty easily and automated.

Lets assume that your Jenkins server url is http://yourjenkins:8080

The details of the last successful build of your appjob can be viewed at http://yourjenkins:8080/job/appjob/lastSuccessfulBuild/

If you visit http://yourjenkins:8080/job/appjob/lastSuccessfulBuild/api/  you can view the details about accessing the details via XML or JSON.

To view the details in XML, visit http://yourjenkins:8080/job/appjob/lastSuccessfulBuild/api/xml

This url provides all job info in a handy XML format. We are interested in the <relativePath>  nodes in the XML document as this provides the path to the artifacts that we are interested in.

We can use the tree  query parameter to filter this information. Change the url to http://yourjenkins:8080/job/appjob/lastSuccessfulBuild/api/xml?tree=artifacts[relativePath]  and you will see that the document now contains relativePaths only. If you change the parameter to tree=artifacts[*] , you will see all artifact information i.e fileName and the relativePath. For our example, we will focus on relativePath only.

Using wget  we can download this XML document and pipe it to xpath  to extract the relativePath information.

You can use curl  instead of wget  and xmllint  or any other tools instead of xpath  too.

wget -q -O- http://yourjenkins:8080/job/appjob/lastSuccessfulBuild/api/xml?tree=artifacts[relativePath] | xpath '//relativePath/text()' 2>&1

This provides the relativePath node information as below:

Found 4 nodes:
-- NODE --
xml2selenium-bestpractices-testcases/target/app1.zip-- NODE --
xml2selenium-bestpractices-testcases/target/app2.zip-- NODE --
xml2selenium-bestpractices-testcases/target/app3.zip-- NODE --
xml2selenium-bestpractices-testcases/target/app4.zip

The — NODE —  information can be stripped out using sed  as below:

wget -q -O- http://yourjenkins:8080/job/appjob/lastSuccessfulBuild/api/xml?tree=artifacts[relativePath] | xpath '//relativePath/text()' 2>&1 | sed -re 's/-- NODE --//g'

The output becomes

Found 4 nodes:

xml2selenium-bestpractices-testcases/target/app1.zip
xml2selenium-bestpractices-testcases/target/app2.zip
xml2selenium-bestpractices-testcases/target/app3.zip
xml2selenium-bestpractices-testcases/target/app4.zip

The response line in the beginning can be removed by the tail  command

wget -q -O- http://yourjenkins:8080/job/appjob/lastSuccessfulBuild/api/xml?tree=artifacts[relativePath] | xpath '//relativePath/text()' 2>&1 | sed -re 's/-- NODE --//g' | tail -n+3

The output becomes

xml2selenium-bestpractices-testcases/target/app1.zip
xml2selenium-bestpractices-testcases/target/app2.zip
xml2selenium-bestpractices-testcases/target/app3.zip
xml2selenium-bestpractices-testcases/target/app4.zip

And finally, the files can be downloaded using a loop

url=http://yourjenkins:8080/job/appjob/lastSuccessfulBuild
wget -q -O- $url/api/xml?tree=artifacts[relativePath] | xpath '//relativePath/text()' 2>&1 | sed -re 's/-- NODE --//g' | tail -n+3\
| while read fileName;\
do \
   wget "${url}artifact/${fileName}";\
done

2 Comments

Leave a Reply

Your email address will not be published. Required fields are marked *