The Jenkins Tutorial Needed to Build, Deploy, Support, and Rollback in 5 Edition

Question

I am new to Jenkins and have some kind of understanding, but need more.

I have a PHP application in the Git repository that uses Composer, there are Assets, the user has downloaded media files, uses Memcache / Redis, has some agents / workers and has migration files.

So far, I realized that I need to create two works in Jenkins.

Job 1 = Creation
Job 2 = Deployment

In the Build job, I install the Git repository as the source, and I install the mail shell script, which has one line composer update .

1) My first question is about how / where files are cloned. I understand that there is a workspace, and every time it is cloned, or only new things are pulled.
2) the composer's update seams to download the same material over and over again, and it looks like it is not cached with multiple builds. I would like to hear an opinion here, but I expected that at the next build he would check the changes and get only the difference. A complete composer update takes several minutes.

In a Deploy job, I would like to set up a process that uses the latest stable build and moves the files to a dedicated folder, such as releases2 . Then several execution scripts are launched, and at the end it updates the / htdocs symbolic folder in the new releases2 folder, so the web server starts serving from this folder on the website.

3) How can I get the latest assembly (in the assembly folder I saw only a couple of log and xml files, could not find the files from git) and go to the new destination.
4) How to configure the destination so that I can save media files between different deployments.
5) When should I deal with Assets (for example, publishing on CDN) after a successful build and before the deployment is complete. Will it be a pre / post hook or other job.
6) When should I clear the caches (memcache, redis).
7) How to roll back to previous versions? And how can I customize the last 5 successful releases.
8) How can I get emails with a build error and failed to deploy email notifications?
9) How operations receive a list of messages sending messages after successful deployment by email.

I noticed that Jenkins has a lot of plugins. Not sure if they are handled by these plugins, but feel free to recommend whatever is done. I also read about Fing, but I'm not sure what it is, and whether I should use it.

I understand that there are a lot of questions in this topic, but if you know the answer for several of them, please send an answer

Answer 1

warning tl; tr

Ok, you want everything. Many questions - a long story.

---

Jenkis is a "just" continuous integration server.

Continuous integration basically means that you don’t need to run the compilation and unit testing phase on the developer's machine, but move this to the central server, right? Since compilation and binding are now located on a central server, the developer has more time to develop, rather than waiting for the compilation to complete. How it started with CI.

Now, looking at PHP projects, there is no compilation or linking process. The work of continuous integration working on PHP projects comes down to simple unit testing and, possibly, generating some reports. You can clearly see this by looking at supporting projects such as Jenkins-PHP, which proves setting up a template for PHP projects on Jenkins - http://jenkins-php.org/example.html

The starting point is "Jenkins does something after you have typed the source." You already have a configuration for your Git repository. It is monitored, and whenever a new commit arrives, a new "build process" is launched.

What is this “build process"?

The build process can be partially configured in the Jenkis GUI. In part, this means that the focus is on the configuration of “triggers” and “notifications,” as well as “report generation.” Reporting means that when certain build tools have completed their tasks, and their log files are processed and converted into a more convenient format.

eg. when phpunit is done, you can use the code coverage log to turn it into a nice HTML page and move it to the / www folder for general viewing.)

But , most of the real work of this build process is described in the build configuration file. This is where building tools come into play, such as "Phing", "ant" (big brother of the finger) and "nant" (victory).

The build tool provides the basis for scripting tasks. Automation is happening here! You will have to script to complete the automation steps yourself. Jenkins is just a graphical interface that provides a few buttons to display build.log and reports and restart the build, correctly.

Or in other words: you cannot just combine Jenkins and your PHP project, hoping that you can click on the build and deployment process along with the GUI. While we are gone! Tools are getting better, but it's a long way to go.

---

Let me tell you a little about Janice. Let's focus on the construction steps.

How will you create and deploy your project when you are only in the CLI? Do it! You might want to write down all the commands and steps associated with a simple text file. Now include these steps in the automation steps. Create "build.xml" in the root folder of your project.

 <?xml version="1.0" encoding="UTF-8"?> <project name="name-of-project"> ... build steps .. </project> 

Now we need some assembly steps. Assembly tools refer to them as "target." Define assembly target groups. You can accomplish each goal on your own, and you can also link them.

 <?xml version="1.0" encoding="UTF-8"?> <project name="name-of-project" default="build"> <target name="build"> <!-- tasks --> </target> <target name="deploy"> <!-- tasks --> </target> </project> 

Rule: keep goals small - no more than 5-7 cli teams in one goal.

Now let's enter the target chain with the dependencies. Suppose your build task should run phpunit before. In the CLI, you just run phpunit and then your build commands. Inside the assembly configuration, you must wrap the calls in exec tasks. This way you create the target "phunit" and add it as a dependency on the target "build". Dependencies are executed before the target is specified.

 <?xml version="1.0" encoding="UTF-8"?> <project name="name-of-project" default="build"> <target name="phpunit" description="Run unit tests with PHPUnit"> <exec executable="phpunit" failonerror="true"/> </target> <target name="build" depends="phpunit"> <!-- tasks --> </target> <target name="deploy"> <!-- tasks --> </target> </project> 

A build tool such as Phing provides many basic tasks such as chown, mkdir, delete, copy, move, exec (...) and additional tasks (for git, svn, notify). See Phing Documentation http://www.phing.info/docs/master/hlhtml/index.html or Ant http://ant.apache.org/manual/

The good thing with Phing is the ability to write AdhocTasks in PHP inside the build configuration file and run them. It is also possible with ant, just create exec tasks that execute PHP and script.

OK - allows you to quickly redirect: you re-created complete assembly and deployment procedures within this assembly configuration. Now you can use the target commands autonomously. Now we go back to Jenkins CI or any other CI server and configure it to run the build tool with the targets. Usually you will have a default goal called main or build , which combines all your goals (steps).

Now, when a new commit arrives, Jenkins begins the build process by building the script.

---

Given these pieces of information about how Jenkins interacts with the build tool, some of your questions are self-evident. You just need to create steps to do what you want ...

Run Q & A round:

Q1: Jenkins workspace folder

A workspace is a place where projects live. New commits arrive there. In the "Advanced" section, you select the working directory for projects without changing the Jenkins home directory. Check the "Use custom workspace" box and select the directory in which Jenkins will extract the code and be embedded in it. It is also possible to configure assembly folders and the number of assemblies there.

Q2: Composer composer stores a local cache - it lives in $COMPOSER_HOME/cache . The local cache will be used when using the same dependencies. This avoids reloading. If a new dependency is introduced or the version has changed, then everything will be extracted and reused on composer install and composer update .

Composer installations / updates are always fresh from the network or cache. There is no support in the vendor folder. The dependency is removed and reinstalled. If it takes a long time, it will take a long time. The end of the story.

If this takes a long time, use Composer one-time, and then add the new assembly targets "zip-copy-vendor-folder" and "copy-unzip-vendor-folder". I think you can imagine what it is. Now you need to enter an if check for the zipped vendor file. If the provider zip file exists, you will skip the composer installation goal and continue with "copy-unzip .." .. well, you got it. This is the setting ... do it only if your dependencies are fairly stable and far from change.

In general, you will need the build target "get-dependencies-with-composer", which runs composer install . The cache will be automatically used by the composer.

Q3: Get the latest build and go to the new destination

The last assembly is in the assembly folder - or, if you have defined a step for moving the file, it is already in your desired folder.

Q4: how to get media files in

Just add the build target to copy media folders to the project.

Q5: add build goals for resource management

You already know the position: it is "after assembly." This means that these are deployment steps, right? Add a new target to upload your folder, possibly via FTP to your CDN.

Q6: when should I clear the caches (memcache, redis)

I suggest going with a simple one: deployment strategy - flash cache-rewarm caches.

Hot calling PHP applications is tricky. You must have a PHP class that supports modifying basic components while the system runs two versions. Old versions disappear from the cache, new versions disappear. Ask this question separately! It is not as easy as one might think. This complicated one of Rasmus Lerdorf's favorite topics.

Q7.1: How to roll back to previous versions?

By running the deployment target / tasks in the folder of the previous version.

Q7.2: And how can I customize the last 5 successful releases.

Jenkins has an option for “how many assemblies to save” in the assembly folder. Set the value to 5.

Q8: How can I get emails with a build error and failed to deploy email notifications?

Automatically. Email notifications by default. If I'm wrong, check out the email alerts.

** Q9: How operations receive a list of messages sending messages after successful deployment by email. **

Add the build target "send-git -log-via-email-to-operations".

---

I feel like I wrote a book today ...

Answer 2

I have no answers to all but a few that I have:

1) My first question is about how / where files are cloned. I understand that there is a workspace, and every time it is cloned, or only new things are pulled.

You correctly understand that the files are cloned in the workspace. However, if you want, you can customize your own workspace by setting it to Advanced Project Options (enable Use custom workspace ), which is located just above the "Source Control" section.

2) the seams of the composite update for reloading the same material, and it seems that it is not cached with multiple assemblies. I would like to hear an opinion here, but I expected that at the next build he would check the changes and get only the difference. A complete composer update takes several minutes.

I have no idea about Composer, but if this thing is also extracted from Git, then Shallow clone might be what you are looking for. This is present in the section: Source Code Management section > Git > Additional Behaviours > Advanced clone behaviours

In the Deploy job, I would like to set up a process that uses the latest stable build and moves the files to a dedicated folder, such as releases2. Then, several execution scripts are launched, and at the end it updates the / htdocs symbolic folder in the new release2 folder, so the web server starts serving from this folder on the website.

I'm not sure why you need separate work for deployment. Everything that you said above can be done in the same work that I assume. In the Build section (just above the "Post-Build Actions" section), you can specify your own script (Win batch / bash / perl / ...) that will perform all the actions necessary for a stable build that is simply created.

3) How can I get the latest build (in the build folder I saw only a couple of log and xml files, could not find the files from git) and go to the new destination.

From your description, I'm pretty sure that you don't have a master slave set up for Jenkins. In this case, the easiest way to locate files received from Git is to check the "Console exit" of the last build (or any build, for that matter). In the first two or three lines of the console output, you will see the path to the assembly workspace. For example, in my case, it is something like:

 Started by timer Building remotely on Slave1_new in workspace /home/ec2-user/slave/workspace/AutoBranchMerge 

4) How to configure the destination so that I can save media files between different versions.

I'm not quite sure what you are looking for. It seems like you need to process your script. Please specify.

5) When should I deal with Assets (for example, publishing on CDN) after a successful build and before the deployment is complete. Should it be a pre / post hook or another job.

If you mean artifacts, then you should check for "Post-Build Actions." It has several options, such as "Archive artifacts", "[ArtifactDeployer] -" Deploy artifacts from the workspace to remote repositories ", etc.). The number of such parameters that you see in this section depends on the number of settings you set plugins.

One useful artifact plugin is https://wiki.jenkins-ci.org/display/JENKINS/ArtifactDeployer+Plugin

6) When should I clear the caches (memcache, redis).

Sorry, I don’t know about this.

7) How to roll back to previous versions? And how can I customize the last 5 successful releases.

Previous version of what? The assembly is always overwritten in the workspace; Only magazines from past collections will be available for viewing. You will need to explicitly install a mechanism (script) to backup the assembly. Also check out this Discard Old Builds section at the top of the project configuration page. Several plugins that you can install are also available. They will help you set up assemblies to save, delete, etc.

8) How can I get emails with a build error and failed to deploy email notifications?

This option is available in Post-build Actions . There is an "Email Notification" that provides basic functionality. If you need better features, I suggest you install the "Email-ext" plugin.

https://wiki.jenkins-ci.org/display/JENKINS/Email-ext+plugin

You can search for hundreds of plugins by going to the Jenkins > Manage Jenkins > Manage Plugins > Available

9) How operators receive a list of messages sending messages after successful deployment by email.

This feature may not be available through the plugin. I believe that some scripting efforts will be needed. This may help you: How do I include Git changelog in Jenkins emails?