Wikimedia Apps/Team/iOS/CI/ProjectSetup

The following documentation will walk you through setting up an individual iOS project for continuous integration.

What To Expect

These instructions use Jenkins and Fastlane as the primary tools to get CI up and running for your project.

The main role of Jenkins is that of scheduling and polling your repository and then kicking off Fastlane to perform the actual build tasks. Additionally Jenkins will be where you view build reports and the first place you look if a build breaks.

Before you Start

Server Access

You will need to access your server, of course. For Wikimedia you can get that information on the Server Access page.

For anyone else, you can just access it directly, use SSH, or the Apple Screen Sharing.app

it is recommended that you use a dedicated machine - like a Mac mini - that you can access over the public internet

Server Setup

If your server has not been setup for CI, first check out the documentation on setting up the server before you begin setting up your project. Any required dependencies for the following documentation will be installed using that guide first.

Setup Git SSH Keys (Optional)

If you need to push changes to Gerrit or Github, you will need to create an SSH Key and upload it to a user with push permissions for the repository.

You can find instructions here to accomplish this here:

Create a Github Account for your Server (Optional)

Similar to the iTunes account. Your build job can include actions like creating git tags, committing build numbers, or commenting on pull requests. You will want the server to have its own account to do this.

Clone Your Project (Optional)

Generally when setting up CI, you will be doing a lot of fiddling to get settings just right. To make this task easier, it is recommended that you “manually” clone your project so you can tweak settings and test without involving Jenkins.

cd ~/Projects
git clone <Your Repo Info>

Setup your Xcode Project

Before you begin, you will (likely) need to make some modifications to your Xcode project. There are many options here, this section will cover one methodology.

The following steps will generate unique bundle ids for each build configuration. This way you can select a bundle id at build time without modifying the project file/info.plist

Create a new Build Configuration called “Beta”.
Create a new Scheme Called “Beta”, change it to use the “Beta” build configuration for all build types, and marked it as a shared scheme.
Add a new Custom Build Variable called BUNDLE_ID_SUFFIX.
Set the BUNDLE_ID_SUFFIX for the Beta config to “.beta”
Commit and push these changes to your repository.

Setup Fastlane

Fastlane works by defining “lanes” for each of your build types. You can define different build actions for each lane.

Below we will describe how to configure the most common options for a build job. Many other options for Fastlane are available. For the full documentation visit the Fastlane website.

Though we will be using Fastlane with Jenkins, you can still run the Fastlane build commands on your local machine and they should work the same way (unless they depend on Jenkins environment variables). Fastlane makes your build process completely portable.

Before you Start

Use the manually cloned project

It is recommended that you use the project you cloned into the “Projects” folder for this part of the setup. You can do these steps on your local machine and then push the changes, but at some point you will need to be working on the server to test your changes.

Fastlane itself is a set of tools written in ruby. You don’t need to be an expert in ruby, but if you know some, you will be able to run arbitrary ruby code in your build steps.

Keychain Access

Keychains remain locked when accessing over ssh. If you perform any Fastlane actions that require keychain access, they will fail unless you are screen sharing and use the terminal on the build machine. (There are other options, but they aren't always secure).

init Fastlane

Open the project directory in terminal
Enter “fastlane init” and follow the instructions to create a basic Fastfile

At this point, you will have a new “fastlane” directory with several files in it. Below we will cover what files you will customize to get your project up and running.

Before you get to set up your build process, you need to define some information

Configure the Appfile

Add a developer account to the Appfile (optional, but required for some actions)

Some build steps (e.g. creating provisioning profiles, submitting apps for App Store Review) will require an Apple ID to log in to the Developer Portal and iTunes Connect. To make available to fastlane, add the following line to the Appfile:

apple_id “you@yourdomain.com”

For security, you may consider injecting these at runtime as environment variable from Jenkins. This way account user names are not committed to version control.

Developer Team id and name (optional)

If the Apple ID that you add to the file belongs to multiple teams, you will need to specify the team you are distributing to.

team_id “XXXXXXXX”
team_name “ACME Corp"

Password

Fastlane will prompt you to enter the password for the Apple ID the first time you run the cert action. Fastlane then stores the password in the keychain so it will be secure (only people who can log into the machine will have access to the password).

Since you may be storing your password on a shared machine, It’s recommended that you create a Developer Portal and iTunes Connect account specifically for your server. This way you can revoke access for the server if you have a security problem without compromising your own account. You can also restrict the permissions of the server account since it doesn’t need administrator privileges.

Per lane settings (optional)

You can wrap any settings in the Appfile within a for_lane block. This allows you to have per lane settings. One thing this is useful for is if you have both an App Store and Enterprise Distribution account that you use to distribute your app.

for_lane 'ios beta' do
   apple_id “you@yourdomain.com”
end

Configure the Deliverfile

Set Environment Variables for ipa paths

You need to make ipa files that you generate available to other actions in fastlane (like uploading to the app store). To do this add the following lines:

ipa ENV["IPA_OUTPUT_PATH"]
beta_ipa ENV["IPA_OUTPUT_PATH"]

This may seem a bit strange, but this is an artifact of Fastlane being a collection of several different tools. This will likely be streamlined and become unnecessary in a future version.

Configure the Fastfile

From here you will be editing the Fastfile to perform specific actions depending on your needs.

Create a lane for your build

A build process is defined by a “lane” in Fastlane. You can create multiple lanes in your Fastfile. Your newly created Fastfile will contain some starter lanes. Feel free to remove, rename, and add lanes to meet your needs.

A simple lane looks like this:

lane :beta do
        sigh
        ipa
        deliver
end

What lanes do I need?

You can create and name lanes for anything you wish. One methodology is to name lanes to map to your build type. Some examples:

beta
appstore
test

Add actions to your lane

Below are the common steps you would add to a lane. You can read about more actions and in-depth options in the Fastlane Actions documentation

Check that the repo is clean

ensure_git_status_clean
This checks that the repo is in a pristine state. This is a good check to run after reseting the repo and before any do any build actions

Reset the Repo

reset_git_repo :force
This resets the repo to pristine state. Any uncommitted work will be destroyed. This is good to run at the beginning of a job to make sure you are not working with any unsaved files.

Install Pods

cocoapods
Add this action to perform a “pod install”.

Requires Cocoapods to be installed. See the Server Setup Guide for details

Increment the build number

increment_build_number
This increments the build number. You can also pass a build number explicitly. This uses agvtool under the hood, so the the normal project setup applies.

Set the build number to the Jenkins build number

increment_build_number ENV["BUILD_NUMBER"]
If you do this, you can match the build number of any app to the Jenkins build number which created the app.

Download Developer Certificates

cert
This downloads any certificates needed for distribution. This command will NOT download the private key. So you still need to manually import the certificate with the private key for the first time. But afterwards, this will download any updated certs.

Requires Apple Developer Portal Account in the Appfile (see instructions above)

Create / Download Provisioning Profile

sigh
This instruction will attempt to download the provisioning file matching the given app identifier, and if not available, will create it for you.

sigh(
      provisioningname: 'TF Alpha',
      appidentifier: 'org.wikimedia.wikipedia.tfalpha',
)

If you are using Apple TestFlight do NOT pass adhoc: true . The new Testflight uses App Store provisioning profiles, even for beta builds.

Requires Apple Developer Portal Account in the Appfile (see instructions above)

Run Unit Tests

xctest
This command is essentially an alias for xcodebuild with the test flag and xcpretty.

You can specify the same arguments as you could with xcodebuild (such as scheme, configuration, or destination). With the destination argument, not only can you pass the platform, device, and OS version, you can also test on actual devices by specifying their name (as seen in the Xcode build destination menu). See Apple documentation for more details.

xctest(
   workspace: "MyProject.xcworkspace",
   scheme: "BetaScheme",
   configuration: "Beta",
   destination: "platform=iOS Simulator,name=iPhone 5s,OS=8.2",
   reports: [
        {
          report: "html",
          output: "build/reports/unit-tests.html"
        },
        {
          report: "junit",
          output: "build/reports/unit-tests.xml"
        }
      ],
)

Test Reports

For xcpretty, you can specify reports. If you do this, reports will be generated for your unit tests that you can display in Jenkins. See the Jenkins section below for instructions on how to import them.

Build an ipa for deployment

ipa
Add this command to generate a build. This command takes similar arguments as xctest. The build will automatically use the provisioning profile created/downloaded in the sigh command above as long as the bundle id matches the provision profile. There are lots of options for the ipa command so be sure to check out the Fastlane Actions documentation.

ipa(
 scheme: 'BetaScheme',
 configuration: 'Beta', 
 clean: nil, # this means 'Do Clean'. Clean project before building.
 archive: nil, # this means 'Do Archive'. Archive project to the Xcode organizer after building.
 verbose: nil, # this means 'Do Verbose'.
)

Currently there is a known bug that requires you to pass the configuration even if you pass the scheme, otherwise the ipa command will always run the “release” scheme.

Upload to Hockey App

hockey
This command can take any options that can be passed to the hockey API. The ipa generated by the ipa command will automatically be passed to the hockey command.

hockey(
 api_token: 'xxxxxxxxxxxxxxxxxxxxxxxxxx',
 notes: "Changelog",
 notify: 0, #Means do not notify
 status: 1, #Means do not make available for download
)

Upload to Apple Testflight

deliver skip_deploy: true, beta: true
This command will upload to the Apple TestFlight service. Passing skip_deploy will make sure it doesn't get submitted for external testing.

Requires that the ipa file path be specified in the Deliverfile. See the Deiliverfile section above.

Upload to Appstore

deliver :force
TBD

Test Fastlane

At this point you should have a lane working in Fastlane. To verify that it works cd to the project directory and run the following command in the terminal: fastlane beta

Check the output in the terminal. If this is your first time running the command, you may be prompted to enter some passwords as noted above.

If you run into issues, you will likely get a stack trace in the terminal. Start investigating there an be sure to check out the Fastlane issues on Github.

Configure Jenkins

At this point your build process is defined in Fastlane, but now you need to configure Jenkins to trigger your builds. You should have already installed Jenkins and made sure you it is accessible via the web browser. If not, check out the CI Server Setup documentation.

install Jenkins Plugins

Depending upon the source control you use for your project, you will may to install Jenkins plugins. Some of these may be pre-installed.

Required Plugins

Git (You are using Git, aren’t you?)

Recommended Plugins

Gerrit
Github

Plugin Installation

Open your browser and open the Jenkins dashboard

If you are working locally, it is probably: http://localhost:8080/. If not, you need to get the URL you specified during Server Setup.

Click on “Manage Jenkins”
Click on “Manage Plugins”
Click “Installed” to see if the plugin you want is already installed
- If so, you are done!
Click on “Available” to view plugins that can be installed
Search for the plugin you want by entering the name in the “Filter” field
Click the checkbox for the plugin you want to install
Repeat steps 7 and 8 for every plugin you need to install
Click “Install Without Restart”

Create a Jenkins Job

Now you need to create job for your project.

Generally you will create one Jenkins job for every type of build. You may have one build for distributing nightly builds, one for running unit tests on specific branches, one for distributing release candidates, etc…

You will repeat the instructions in this section for each type of build.

Create a new Project

Open your browser and open the Jenkins dashboard
Click “New Item”
Enter a name for your project like “My App Beta”
Select “FreeStyle Project”
Click “OK”

You can also “Copy an existing project” if you have a similar one already configured.

At this point you will be viewing the Project Configuration.

The next few sections will explain how to view, edit, and save your Project. These steps will be performed very frequently during the setup process.

Viewing your Project

From here on, you can view your job by doing the following: 1. Open your browser and open the Jenkins dashboard
2. Click on the Project you wish to view.

On the project page you will see the available options, project status, and previous builds.

Editing your Project configuration

When viewing your project, you can make changes by doing the following: 1. View project using the instructions above
2. Click “Configure”
3. Make your changes
4. Save your changes by:
- Clicking “Apply” to save changes but keep editing
- Clicking “Save” to save changes and stop editing

Testing your Project Configuration

You can build your project at any time by doing the following:
1. View project using the instructions above
2. Click “Build Now” in the side bar.
3. A build will appear in the left sidebar (if it doesn’t refresh the page)

Viewing build logs and reports

You can see the results of a build (even while in progress) by doing the following:
1. View project using the instructions above
2. Click on the build you wish to view in the “Build History” sidebar

From here you can view the console output (log), test results, git change sets, etc…

Enable cloning of your project

After you create a project and understand how to edit it, your first step is configuring the projects SCM settings so it can download your code from the repository.

Open your Project configuration by using the instructions above
Go to the “Source Code Management Section” and select “Git”
Enter your “Repository URL”
- If you need only pull access, you can just use http
- If you need to push changes, you should use SSH set up SSH keys as mentioned above
Enter the branch you wish to build
Save your changes using the instructions above

To test your SCM setup, click “Build Now” in the side bar. A build will appear in the left sidebar.

If your project was successfully cloned, you will see a blue dot for the build, if not you will see a red dot.

If your build failed you will need to troubleshoot your git configuration. Follow the instructions under “Viewing build logs and reports” and open the “Console Output” to see the build log for clues on why it failed.

Build Triggers

You can use various triggers to kick off your builds. Generally you use one trigger per job, but you can use multiple triggers if needed. Check the boxes for the triggers you want to use and click “Apply” after you configure the trigger.

While configuring your project you can safely uncheck these boxes without loosing your settings so builds are not being attempted during the setup process.

Below are some short descriptions on how and why to use various triggers.

Build Periodically

Use for jobs like nightly builds. These will run, even if no new commits we made. Schedules are defined using cron syntax. Jenkins provides syntax help for setting this up.

Gerrit Trigger

This is available if you install the Gerrit Plugin. This will enable you to build each time a new patch set is submitted for review.

Poll SCM

This allows you to poll the git repo and check for new commits. If new commits are found, a new build will begin. The polling schedule is defined using cron syntax just like the “Build Periodically” option.

Colors (Optional)

If you installed the ANSI Color Plugin, you can enable console colors by checking the “Color ANSI Console Output” box under “Build Environment”. This makes your logs much easier to read.

Execute the Fastlane tool

Open your project configuration
Click “Add build step” and select “Execute Shell” from the drop down
Enter the following in the text box:
export LANG=en_US.UTF-8
fastlane beta
Save your changes

Import Fastlane XML reports into Jenkins

Open your project configuration
Click “Add post-build action” and select “Publish JUnit test result report” from the drop down
Enter “fastlane/report.xml” in the “Test report XMLs” text field
- If you are performing unit tests append “, build/reports/junit.xml” to the above line
Save your changes

These paths are based on the out of box Fastlane configuration. Your report locations may be different depending on your Fastlane configuration.

Import Fastlane HTML reports into Jenkins

Open your project configuration
Click “Add post-build action” and select “Publish HTML reports” from the drop down
Add “build/reports/” in the “HTML directory to archive” field
Add “tests.html” in the “Index pages” field
Add a name like “Beta Unit Tests” to the “Report title” field
Save your changes

These paths are based on the out of box Fastlane configuration. Your report locations may be different depending on your Fastlane configuration.

Try it out!

Jenkins and Fastlane are now configured. You can test your configuration by running a manual build as described above. Or by tripping one of the triggers you set up.