msgbartop
MAC OS X, Linux, Windows and other IT Tips and Tricks
msgbarbottom

05 Jun 15 Installing GitLab (MySQL based) on Ubuntu 14.04.2 LTS Server

Note: Instructions based on but have been modified in a few places to make it work with mySQL:
//gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/install/installation.md
and //gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/install/database_mysql.md

Extra related Info:
//gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/install/installation.md
//www.linode.com/docs/applications/development/gitlab-on-ubuntu-14-04
and //www.digitalocean.com/community/tutorials/how-to-set-up-gitlab-as-your-very-own-private-github-clone#database-setup

STEPS

adduser --disabled-login --gecos 'GitLab' git
apt-get install -y build-essential zlib1g-dev libyaml-dev libssl-dev libgdbm-dev libreadline-dev libncurses5-dev libffi-dev curl openssh-server redis-server checkinstall libxml2-dev libxslt-dev libcurl4-openssl-dev libicu-dev logrotate python-docutils pkg-config cmake nodejs
apt-get install postfix git libpq-dev sudo nodejs

(Select ‘Internet site’ for postfix)
# Make sure ruby is de-installed (we need the manually installed version >2.0 for Gitlab)
apt-get remove ruby
mkdir /tmp/ruby && cd /tmp/ruby
wget //ftp.ruby-lang.org/pub/ruby/2.1/ruby-2.1.2.tar.gz
tar xvzf ruby-2.1.2.tar.gz
cd ruby-2.1.2
./configure --without-X11 --disable-install-rdoc --prefix=/usr/local
make && make install

Installing Mysql Server

Notes: These instructions are based on //gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/install/database_mysql.md
FYI:
The above site mentions the Mysql Bug (//bugs.mysql.com/bug.php?id=65830) but it has been fixed in MySQL Ver. 5.5.24
Install the database packages
apt-get install -y mysql-server mysql-client libmysqlclient-dev
Ensure you have MySQL version 5.5.24 or later
mysql --version
# Pick a MySQL root password (can be anything), type it and press enter
# Retype the MySQL root password and press enter
# Secure your installation (not really needed in this set-up if the server for for internal use)
mysql_secure_installation
# Login to MySQL
mysql -u root -p
# Type the MySQL root password

# Create a user for GitLab
Note: do not type the ‘mysql>’, this is part of the prompt
# change $password in the command below to a real password you pick
mysql> CREATE USER 'git'@'localhost' IDENTIFIED BY '$password';
# Ensure you can use the InnoDB engine which is necessary to support long indexes
# If this fails, check your MySQL config files (e.g. `/etc/mysql/*.cnf`, `/etc/mysql/conf.d/*`) for the setting “innodb = off”
mysql> SET storage_engine=INNODB;
# Create the GitLab production database
mysql> CREATE DATABASE IF NOT EXISTS `gitlabhq_production` DEFAULT CHARACTER SET `utf8` COLLATE `utf8_unicode_ci`;
# Grant the GitLab user necessary permissions on the database
mysql> GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, INDEX, ALTER, LOCK TABLES ON `gitlabhq_production`.* TO 'git'@'localhost';
# Quit the database session
mysql> \q
# Try connecting to the new database with the new user
sudo -u git -H mysql -u git -p -D gitlabhq_production
# Type the password you replaced $password with earlier
# You should now see a ‘mysql>’ prompt
# Quit the database session
mysql> \q

Installing REDIS

apt-get install redis-server
# Configure redis to use sockets
cp /etc/redis/redis.conf /etc/redis/redis.conf.orig
# Disable Redis listening on TCP by setting ‘port’ to 0
sed 's/^port .*/port 0/' /etc/redis/redis.conf.orig | sudo tee /etc/redis/redis.conf
# Enable Redis socket for default Debian / Ubuntu path
echo 'unixsocket /var/run/redis/redis.sock' | sudo tee -a /etc/redis/redis.conf
# Grant permission to the socket to all members of the redis group
echo 'unixsocketperm 770' | tee -a /etc/redis/redis.conf
# Create the directory which contains the socket
mkdir /var/run/redis
chown redis:redis /var/run/redis
chmod 755 /var/run/redis

# Persist the directory which contains the socket, if applicable
if [ -d /etc/tmpfiles.d ]; then echo 'd /var/run/redis 0755 redis redis 10d -' | tee -a /etc/tmpfiles.d/redis.conf ; fi
# Activate the changes to redis.conf
service redis-server restart
# Add git to the redis group
usermod -aG redis git

Installing GitLAB

cd /home/git
git clone //gitlab.com/gitlab-org/gitlab-ce.git -b 7-11-stable gitlab

# Give the ownership to git user of transferred repository
chown -R git: /home/git/gitlab
cd /home/git/gitlab

# Make sure GitLab can write to the log/ and tmp/ directories
chmod -R u+rwX {log,tmp,tmp/pids,tmp/sockets,public/uploads}
chown -R git log/
chown -R git tmp/

# Create the GitLab config file:
sudo -u git -H cp config/gitlab.yml.example config/gitlab.yml
nano config/gitlab.yml

# You need to change the value of host to the fully-qualified domain of your server.
# Also set the email_from and support_email to the email addresses intended for GitLab.
# Content of /home/git/gitlab/config/gitlab.yml
production: &base
gitlab:
host: gitlab.server.com
port: 443
true
...
email_from: gitlab@gitlab.server.com
...
support_email: admin@server.com

# Make sure GitLab can write to the log/ and tmp/ directories
chown -R git {log,tmp}
sudo chmod -R u+rwX,go-w log/
sudo chmod -R u+rwX tmp/

# Create directory for satellites
sudo -u git -H mkdir /home/git/gitlab-satellites
chmod u+rwx,g=rx,o-rwx /home/git/gitlab-satellites

# Make sure GitLab can write to the tmp/pids/ and tmp/sockets/ directories
chmod -R u+rwX tmp/pids/
chmod -R u+rwX tmp/sockets/

# Make sure GitLab can write to the public/uploads/ directory
chmod -R u+rwX public/uploads

Configure Unicorn

# Copy the example Unicorn config
sudo -u git -H cp config/unicorn.rb.example config/unicorn.rb
# Find number of CPU cores in order to configure Redis properly
nproc
# Enable cluster mode if you expect to have a high load instance
# Ex. change amount of workers to 3 for 2GB RAM server
# Set the number of workers to at least the number of cores
nano config/unicorn.rb
# Copy the example Rack attack config
sudo -u git -H cp config/initializers/rack_attack.rb.example config/initializers/rack_attack.rb
# Configure Git global settings for git user, used when editing via web editor
sudo -u git -H git config --global core.autocrlf input
# Configure Redis connection settings
sudo -u git -H cp config/resque.yml.example config/resque.yml
# Change the Redis socket path if you are not using the default Debian / Ubuntu configuration
nano config/resque.yml

Important Note:
Make sure to edit both gitlab.yml and unicorn.rb to match your setup.

Note: If you want to use HTTPS, see Using the following HTTPS for the additional steps.
(Also see //gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/install/installation.md#using-https)

To use GitLab with

1. In gitlab.yml:
Set the port option in section 1 to 443.
Set the https option in section 1 to true.
2. In the config.yml of gitlab-shell:
Set gitlab_url option to the HTTPS endpoint of GitLab (e.g. //git.example.com).
Set the certificates using either the ca_file or ca_path option.
3. Alternatively use the gitlab-ssl Nginx example config instead of the gitlab config.
Update YOUR_SERVER_FQDN.
Update ssl_certificate and ssl_certificate_key.
Review the configuration file and consider applying other security and performance enhancing features.

Configure the Database connection:

# Create the config/database.yml file
cp config/database.yml.mysql config/database.yml
# Adapt the file config/database.yml to configure the Database parameters
# Normally only the git user password and host and port need to be changed/added as follows
nano config/database.yml
production:
adapter: mysql2
encoding: utf8
reconnect: false
database: gitlabhq_production
pool: 10
username: git
password: "secure password"
host: localhost
port: 3306
# socket: /tmp/mysql.sock

# Make sure that config/database.yml is readable to git only:
chown git: config/database.yml
sudo -u git -H chmod o-rwx config/database.yml

# Install the gems:
Note : Under ‘N‘ in ‘-jN‘ is the number of CPUs in your server. This helps to accelerate the process.
su -
gem install bundler
exit
su - git
cd ~/gitlab
bundle install -jN --deployment --without development test postgres aws kerberos

Install GitLab Shell

#Install GitLab shell, which is an SSH access and repository management software for GitLab:
bundle exec rake gitlab:shell:install[v1.9.4] REDIS_URL=redis://localhost:6379 RAILS_ENV=production
# Edit the GitLab shell configuration file and make sure of the following content
# and adapt to your needs and environment (especially gitlab_url: the rest should be left as is but just check.)
nano /home/git/gitlab-shell/config.yml
user: git
gitlab_url: //gitlab.server.com/
http_settings:
self_signed_cert: false
repos_path: "/home/git/repositories/"
auth_file: "/home/git/.ssh/authorized_keys"
redis:
bin: "/usr/bin/redis-cli"
namespace: resque:gitlab
socket: "/var/run/redis/redis.sock"
log_level: INFO
audit_usernames: false

# Initialize database and activate advanced features:
Run the following 2 commands as git user:

su - git
cd /home/git/gitlab
bundle exec rake gitlab:setup RAILS_ENV=production

# The command will display the following message
This will create the necessary database tables and seed the database.
You will lose any previous data stored in the database.
Do you want to continue (yes/no)?

# Type yes and press Enter to continue.
# It is important to remember the last 3 lines (Administrator account created:)
login.........root
password......5iveL!fe

# Install the init script and make GitLab start on boot:
sudo cp /home/git/gitlab/lib/support/init.d/gitlab /etc/init.d/gitlab
sudo chmod 755 /etc/init.d/gitlab

# Make GitLab start on boot:
sudo update-rc.d gitlab defaults 21
# Set up logrotate:
sudo cp /home/git/gitlab/lib/support/logrotate/gitlab /etc/logrotate.d/gitlab
# Check application status:
cd /home/git/gitlab
bundle exec rake gitlab:env:info RAILS_ENV=production

# The following information should show up
System information
System information
System: Ubuntu 14.04
Current User: git
Using RVM: no
Ruby Version: 2.1.6p336
Gem Version:
Bundler Version: 1.10.2
Rake Version: 10.4.2
Sidekiq Version: 3.3.0
.
GitLab information
Version: 7.11.4
Revision: b725318
Directory: /home/git/gitlab
DB Adapter: mysql2
URL: //gitlab.mydomain.net
HTTP Clone URL: //gitlab2.mydomain.net/some-project.git
SSH Clone URL: git@gitlab2.mydomain.net:some-project.git
Using LDAP: yes
Using Omniauth: no
.
GitLab Shell
Version: 2.6.3
Repositories: /home/git/repositories/
Hooks: /home/git/gitlab-shell/hooks/
Git: /usr/bin/git

# Compile assets:
bundle exec rake assets:precompile RAILS_ENV=production
# Configure Git global settings for the git user:
git config --global user.name "GitLab"
git config --global user.email "gitlab@mygitlab.server.com"
git config --global core.autocrlf input

Note:
Set the above value for ‘user.email’ according to what is set in config/gitlab.yml

Login back as root superuser
exit
# Start GitLab:
service gitlab start

INSTALL NginX for Gitlab

# Install Nginx if you haven’t installed it:
apt-get install nginx
# Copy the sample site config:
cp /home/git/gitlab/lib/support/nginx/gitlab /etc/nginx/sites-available/gitlab
# Install the web certificates in a /etc/nginx/certs/
# Combine the certificate and CA together in one file
cat /etc/nginx/certs/wildcard.server.com_CRT.pem /etc/nginx/certs/Thawte_2010.02.08-2020.02.07_CA.pem > /etc/nginx/certs/wildcard.server.com_CRT_CA.pem
# Open the config file(/etc/nginx/sites-available/gitlab) and adapt the following configuration entries
listen *:443 ssl;
server_name gitlab.server.com;
ssl_certificate /etc/nginx/certs/wildcard.server.com_CRT_CA.pem;
ssl_certificate_key /etc/nginx/certs/wildcard.server.com_KEY.pem;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers HIGH:!aNULL:!eNULL:!EXPORT:!DSS:!DES:!SSLv2:!MD5;
gzip off;

# Deactivate the default configuration
rm /etc/nginx/sites-enabled/default
# Activate the site and restart Nginx to take effect
ln -s /etc/nginx/sites-available/gitlab /etc/nginx/sites-enabled/gitlab
service nginx restart

# If Nginx failed to start with the following message
Restarting nginx: nginx: [emerg] could not build the server_names_hash, you should increase server_names_hash_bucket_size: 32
# Open /etc/nginx/nginx.conf and uncomment the following line
server_names_hash_bucket_size 64;
Then restart Nginx.

Open GitLab on Your Browser

# Double check the application status:
su - git
cd ~/gitlab
bundle exec rake gitlab:check RAILS_ENV=production

Results:
If most of the items are green and some are purple (which is okay since you don’t have any git project yet), then you have successfully installing GitLab.

# First initialization of Gitlab and Password change
# Type the following address in browser:
//gitlab.server.com
# First thing: Change the password of administrator (admin@local.host)
# Then confirm the new password by entering it twice.
# re-login and Tada !!! BobsYourUncle 🙂

Troubleshooting Gitlab

# Self check command:
cd /home/git/gitlab
sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production SANITIZE=true

# Check the general configuration of GitLab:
cd /home/git/gitlab
sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production

# Check/precompile if all assets were properly pre-compiled or for assets access errors:
cd /home/git/gitlab
sudo -u git -H bundle exec rake assets:precompile RAILS_ENV=production

# INFO: Gitlab Logs are found in:
/home/git/gitlab/log/*

LDS-LDAP Authentication

In order to enable the LDS-LDAP authentication make the following changes:
Edit the file /home/git/gitlab/config/gitlab.yml and modify the section ldap: as follows:
ldap:
enabled: true
host: '10.23.69.11'
port: 636
uid: 'userPrincipalName'
method: 'ssl'
bind_dn: 'lds-auth'
password: '{password}'
active_directory: true
allow_username_or_email_login: false
base: 'DC=CORP,DC=ad,DC=server,DC=com'
user_filter: ''

#Restart gitlab daemon
service gitlab restart
#Check the LDAP authentication mechanism with the following command:
cd /home/git/gitlab
sudo -u git -H bundle exec rake gitlab:ldap:check RAILS_ENV=production

Results:
Checking LDAP ...
LDAP users with access to your GitLab server (only showing the first 100 results)
DN: CN=............
If the first 100 users DN: data is shown then the LSD-LDAP for gitlab is working

Instructions on how to create a new user on GitLab

– Login with your company login in LDAP Authentication at //gitlab.mydomain.net. The new user will automatically be created in GitLab system.
– Add your public SSH key in the page //gitlab.mydomain.net/profile/keys
– Remember your username in the page: //gitlab.mydomain.net/profile/account under ‘Change Username’ field.
– Remember your email in the page: //gitlab.mydomain.net/profile under ‘Email’ field.
– For linux users using the git command line, run the following commands(assuming that you want your git workspace in ~/gitlab/ directory)
Note: Make sure you replace the above remembered username and email below shown as <USERNAME> and <EMAIL>
– Initializing a workspace for gitlab repositories
mkdir ~/gitlab/ ; cd ~/gitlab/
git config --global user.name <USERNAME>

– Then verify that you have the correct username:
git config --global user.name
– To set your email address, type the following command:
git config --global user.email <EMAIL>
– To verify that you entered your email correctly, type:
git config --global user.email
git config --global --list

– Change to simple push default format
git config --global push.default simple
– Create a new repository as new project in the gitlab web interface run the following command to clone the repository in your local git workspace(assuming here ~/gitlab/)
cd ~/gitlab/
git clone git@gitlab.mydomain.net:/<USERNAME>/<PROJECTNAME>.git

– To commit the first file(special push case for the first file):
cd <PROJECTNAME>
echo "first file content" > first_file
git add first_file
git commit -m 'test commit 1'
git push origin master

– Now all the other files will be pushed normally as follows:
echo "second file content" > second_file
git add second_file
git commit -m 'test commit 2'
git push

09 Dec 14 Installing GITLAB-Omnibus in Debian Wheezy

Introduction:

The instructions here have been based on the site: //about.gitlab.com/downloads/.
I did what is shown there but it didn’t work immediately. I had to do the following tweaks and then it all worked fine so far.
IMPORTANT: In order to stay updated see the last par at the end of this article for instruction son how to Upgrade.

In these examples I use the domain example.com. Replace it with your own domain.

Steps

Run the following commands to install GitLab.
apt-get install postfix
wget --no-check-certificate //downloads-packages.s3.amazonaws.com/debian-7.6/gitlab_7.5.3-omnibus.5.2.1.ci-1_amd64.deb
dpkg -i gitlab_7.5.3-omnibus.5.2.1.ci-1_amd64.deb
nano /etc/gitlab/gitlab.rb

Content:
# Check and change the external_url to the address your users will type in their browser
external_url '//gitlab.example.com'
gitlab_rails['gitlab_email_from'] = 'git@gitlab.example.com'

Create the following certificate files:
/etc/gitlab/ssl/gitlab.example.com.crt
/etc/gitlab/ssl/gitlab.example.com.key

Start the last installation procedure:
gitlab-ctl reconfigure
Here is what you see in logs if you don’t make the following changes:
WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1'
to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.

So, edit /etc/sysctl.conf and add:vm.overcommit_memory = 1
Run:sysctl vm.overcommit_memory=1
Make sure your DNS settings of gitlab.example.com are pointing to the IP of this host. Login at : //gitlab.example.com
Login: root
Passowrd: 5iveL!fe

– Change password of Administrator(root)
– Create new user(s)
…. and you’re up and running GitLab.
——————————————————————
Some extra information regarding the administration of GitLab:

The gitlab-ctl Command:
Info: (/usr/bin/gitlab-ctl -> /opt/gitlab/bin/gitlab-ctl)
This command should be run as root or under sudo:
Syntax:
gitlab-ctl SubCommand [ProcessName]
SubCommand Description:
cleanse Delete *all* gitlab data, and start from scratch.
deploy-page Put up the deploy page
graceful-kill Attempt a graceful stop, then SIGKILL the entire process group.
help Print this help message.
hup Send the services a HUP.
int Send the services an INT.
kill Send the services a KILL.
once Start the services if they are down. Do not restart them if they stop.
reconfigure Reconfigure the application.
restart Stop the services if they are running, then start them again.
service-list List all the services (enabled services appear with a *.)
show-config Show the configuration that would be generated by reconfigure.
start Start services if they are down, and restart them if they stop.
status Show the status of all the services.
stop Stop the services, and do not restart them.
tail Watch the service logs of all enabled services.
term Send the services a TERM.
uninstall Kill all processes and uninstall the process supervisor (data will be preserved).

ProcessName valid values:
logrotate
nginx
postgresql
redis
sidekiq
unicorn

For more information on GitLab see:
//doc.gitlab.com/ci/
Use the manual mode to install GitLab shown at:
//www.digitalocean.com/community/tutorials/how-to-set-up-gitlab-as-your-very-own-private-github-clone
Use LDAP to login:
//github.com/gitlabhq/omniauth-ldap
Or used Windows ADS to login:
//doc.gitlab.com/ee/integration/ldap.html
and //gitlab.com/gitlab-org/gitlab-ce/blob/master/config/gitlab.yml.example
The some extra info/troubleshooting:
//gitlab.com/gitlab-org/omnibus-gitlab/blob/master/README.md
//stackoverflow.com/questions/15321532/setting-up-gitlab-ldap-authentication-without-special-gitlab-user
and
//stackoverflow.com/questions/25046982/gitlab-ldap-authentication-settings
And the support Forum:
//groups.google.com/forum/#!forum/gitlabhq

Troubleshooting

Problem:
Reference:

I can push by clone project using ssh, but it doesn’t work when I clone project with https. it shows message error as below.
server certificate verification failed. CAfile: /etc/ssl/certs/ca-certificates.crt CRLfile: none
Solution 1:
You need to check the web certificate used for your gitLab server, and add it to your (git_intallation_folder)/bin/curl-ca-bundle.crt.
To get that certificate (that you would need to ad to your curl-ca-bundle.crt file), run:
echo -n | openssl s_client -showcerts -connect yourGitLabServer:YourHttpGilabPort 2>/dev/null | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p'
To check the CA (Certificate Authority issuer), run:
echo -n | openssl s_client -showcerts -connect yourGitLabServer:YourHttpGilabPort 2>/dev/null | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p'| openssl x509 -noout -text | grep "CA Issuers" | head -1
Findekano adds in the comments: to identify the location of curl-ca-bundle.crt, you could use the command:
curl-config --ca

Solution 2 (not really recommended for production servers):
To check if at least the clone works without checking said certificate, you can set:
export GIT_SSL_NO_VERIFY=1
or
git config --global http.sslverify false

But that would be for testing only, as illustrated in “SSL works with browser, wget, and curl, but fails with git”.

Creating a new repository/project with Linux clients

Example: creating the project gitrepo1

Git global setup
git config --global user.name "Administrator"
git config --global user.email "joe.doe@mydomain.com"

Create a new repository
mkdir gitrepo1
cd gitrepo1
git init
touch README.md
git add README.md
git commit -m "first commit"
git remote add origin git@gitlab.example.com:joe-doe/gitrepo1.git
git push -u origin master

Push an existing Git repository
cd existing_git_repo
git remote add origin git@gitlab.example.com:joe-doe/gitrepo1.git
git push -u origin master

Note: The ownername is the username of the person who created the project. If that username has already a Dot ‘.’ in it, it should be replaced by a Dash’-‘ in the above commands.

Instructions on how to create a new user on GitLab

– Login with your company login in LDAP Authentication at //gitlab.mydomain.net. The new user will automatically be created in GitLab system.
– Add your public SSH key in the page //gitlab.mydomain.net/profile/keys
– Remember your username in the page: //gitlab.mydomain.net/profile/account under ‘Change Username’ field.
– Remember your email in the page: //gitlab.mydomain.net/profile under ‘Email’ field.
– For linux users using the git command line, run the following commands(assuming that you want your git workspace in ~/gitlab/ directory)
Note: Make sure you replace the above remembered username and email below shown as <USERNAME> and <EMAIL>
– Initializing a workspace for gitlab repositories
mkdir ~/gitlab/ ; cd ~/gitlab/
git config --global user.name <USERNAME>

– Then verify that you have the correct username:
git config --global user.name
– To set your email address, type the following command:
git config --global user.email <EMAIL>
– To verify that you entered your email correctly, type:
git config --global user.email
git config --global --list

– Change to simple push default format
git config --global push.default simple
– Create a new repository as new project in the gitlab web interface run the following command to clone the repository in your local git workspace(assuming here ~/gitlab/)
cd ~/gitlab/
git clone git@gitlab.mydomain.net:/<USERNAME>/<PROJECTNAME>.git

– To commit the first file(special push case for the first file):
cd <PROJECTNAME>
echo "first file content" > first_file
git add first_file
git commit -m 'test commit 1'
git push origin master

– Now all the other files will be pushed normally as follows:
echo "second file content" > second_file
git add second_file
git commit -m 'test commit 2'
git push

Upgrading GitLab Omnibus

Here are the instructions on how to upgrade GitLab Omnibus Community Edition.
Reference: //packages.gitlab.com/gitlab/gitlab-ce
curl -s //packages.gitlab.com/install/repositories/gitlab/gitlab-ce/script.deb.sh | bash
apt-get install gitlab-ce=7.14.1-ce.0

And let the screen roll while watching for errors. 🙂