Mysql on Janik von Rotz

Zabbix stack with Vagrant, CoreOS and Docker

Thu, 08 Oct 2015 16:03:44 +0000

This short guide shows you how to set up a virtual Zabbix Network Monitoring Stack with MySQL and CoreOS deployed by Docker.

On your host machine make sure that these tools are available.

First we are going to setup a virtual machine with Vagrant.

Create a project directory and open navigate there with your command line. Next clone the CoreOS Vagrant repository.

git clone https://github.com/coreos/coreos-vagrant/
cd coreos-vagrant

This repository is preconfigured CoreOS installation.

Open up the Vagrant file and edit the lines as showed below.

$vm_memory = 2024

Now install and run the virtual machine with Vagrant.

vagrant up

If this command finishes without an error, your virtual machine should be up and running in the VirtualBox environment.

Now connect to the virtual machine with ssh.

vagrant ssh

Hint: For windows hosts, make sure that the ssh.exe binary that is shipped with Git is available in your system PATH variable.

CoreOS has Docker pre-installed, so no worry about that. We will now deploy our MySQL and Zabbix stack with preconfigured Docker images.

Install and run the MySQL with Docker.

docker run --name mysql -p 3306:3306 -e MYSQL_ROOT_PASSWORD=password -d mysql:latest

Install and the phpMyAdmin image.

docker run -d --name phpmyadmin --link mysql:mysql -p 1234:80 nazarpc/phpmyadmin

Get the configuration of your MySQL container.

docker inspect mysql

And look for the IPv4 address. We will use in the next statement.

Run the Zabbix image

docker run \
-d \
--name zabbix \
-p 80:80 \
-p 10051:10051 \
-link mysql:db \
--env="DB_USER=root" \
--env="DB_PASS=password" \
--env="SLACK_WEBHOOK=webhook_url" \
million12/zabbix-server

Now open the browser on your host machine to http://host_ip and login with Admin:zabbix.

You can get the host machines ip by tipping ifconfig on command line.

You phpMyAdmin installation is accessbile under http://host_ip:1234.

Troubleshooting

In case some fails are doesn’t work as supposed, follow these steps:

Reboot your virtual machine.

sudo reboot

docker start container_name

Check if the containers have been started successfully.

docker ps

Source

https://hub.docker.com/r/million12/zabbix-server/

Build a Java 3-tier application from scratch - Part 1: Introduction and project setup

Sun, 15 Mar 2015 16:57:38 +0000

Welcome to my first post of the Java 3-tier application tutorial. In this tutorial I will tell show how you can develope a Model-View-Controller based Java desktop and server application using the latest tools and frameworks.

This tutorial uses advanced techniques and programming paradigms, make sure to meet at least the following requirements:

MySQL Database design and administration
Eclipse development
Java classes, interfaces and generics
MVC programming concept
ORM techniques
JAX-RS with jackson aka RESTeasy with JSON
Restful webservices
CRUD Data access object
Booo to XML, hail to JSON
Gradle build and task scripts

I also recommend to finish my other tutorial first (if not done yet): Superb Java development with Gradle, Hibernate, FlywayDB, JavaFX and Eclipse Part 1 and Part 2.

To accomplish this tutorial you will need several development tools and frameworks.

I will definitely use these programs:

Eclipse: The development IDE recommended for this tutorial.
MySQL server: OpenSource database server.
Gradle: A task runner and package manager for your Java project (in case you’re a web developer that would be bower and grunt).
JavaFX Scene Builder Tool to create user interfaces.

Architecture

Our strategy for this project is quite simple. EclipseLink will communicate with the database and acts as our ORM, JAX-RS is used to create our JSON restful webservice and JavaFX is the framework for our client app. Below is a picture that shows our Architecture, the commuication flow and where we use 3rd party libraries.

Our application architecture consists of 3 eclipse projects:

client: Thin app to interact with the application
webservice: Communicates with the database and serves the REST interfaces.
common: Components used by both projects.

Project setup

Now we are going to set up these 3 projects with gradle.

Create a project folder /../<project> somewhere on your drive.
To this folder add 3 more directories: common, webservice and client.
In the project folder and within these 3 directories add a build.gradle file.
Add a second file called settings.gradle to the project folder.
Open your command line and navigate to the project folder: cd <project>.
Update the file content as showed below.

settings.gradle

include ":common", ":webservice", ":client"

This file will tell gradle that we work with a multiproject setup.

build.gradle

subprojects {

  apply plugin: "java"
  apply plugin: "eclipse"

  repositories {
    mavenCentral()
  }

  configurations {
    lombok
  }

  dependencies {
     testCompile 'junit:junit:4.+'
     compile 'org.projectlombok:lombok:1.+'
     lombok 'org.projectlombok:lombok:1.+'
  }

  // install lombok for Eclipse

  sourceSets.each{ sourceSet ->
    sourceSet.compileClasspath += configurations.lombok
    sourceSet.ext.delombok = new File(buildDir, "generated-src/delombok/" + sourceSet.name);
  }

  task installLombok() {
    dependsOn configurations.lombok
  } << {
      File jarFile = null;
      configurations.lombok.resolvedConfiguration.resolvedArtifacts.find {
          if ("lombok".equals(it.name)) {
              jarFile = it.file;
          }
      }
      javaexec {
          main="-jar"
          args = [
              jarFile,
              "install",
              "auto"
          ]
      }
  }
  eclipseProject.dependsOn installLombok

  // install lombok for Javadoc

  task delombok() {
    dependsOn configurations.compile
    dependsOn configurations.lombok
  } << {
      File jarFile = null;
      configurations.lombok.resolvedConfiguration.resolvedArtifacts.find {
          if ("lombok".equals(it.name)) {
              jarFile = it.file;
          }
      }
      sourceSets.each{ sourceSet ->
          def classPath = sourceSet.compileClasspath.files.join(";")
          def delombokPath = sourceSet.ext.delombok
          delombokPath.mkdirs();
          javaexec {
              main = "-jar"
              args jarFile, "delombok"
              if (!classPath.empty) {
                  args "-c", classPath
              }
              args "-d", delombokPath
              args sourceSet.allJava.srcDirs
          }
      }
  }

  javadoc {
      dependsOn delombok
      source = sourceSets.main.ext.delombok
  }
}

This is the master gradle build file and contains definition for each subproject. The java and eclipse plugin allows us to import the 3 projects into Eclipse as a Java project. Maven central is our main repository, this is the provider where we download our libraries. Within this project we will make use of project lombok. This project intends to get rid of the implementation process of getter and setter methods by generating them with annotations. You will see in the next chapter how this works.

common/build.gradle

dependencies {
  compile 'org.eclipse.persistence:javax.persistence:2.+'
  compile 'org.eclipse.persistence:eclipselink:2.+'

  compile 'org.codehaus.jackson:jackson-core-asl:1.+'
  compile 'org.codehaus.jackson:jackson-mapper-asl:1.+'
  
  compile 'org.jboss.resteasy:resteasy-jaxrs:3.+'
  compile 'org.jboss.resteasy:resteasy-jaxb-provider:3.+'
  compile 'org.jboss.resteasy:resteasy-client:3.+'
  
  compile 'org.apache.commons:commons-lang3:3.+'
  compile 'com.typesafe:config:1.+'
  compile 'org.slf4j:slf4j-simple:1.+'
  compile 'com.github.javafaker:javafaker:0.+'
}

Both application components going to use these libraries: EclipseLink: ORM for Java. jackson core: Core components of jackson to parse JSON data. jackson mapper: Convert between POJO and JSON. RESTeasy client: client api to consume the webservice. RESTeasy: JAX-RS implementation to develop RESTful client and server applications. Apache Commons: popular and reusable Java components. Typesafe Config: JVM configuration library. Java Faker: library to create fake data.

webservice/build.gradle

apply plugin: 'jetty'

dependencies {
  compile project(':common')
  compile 'mysql:mysql-connector-java:5.+'
  compile 'org.jboss.resteasy:resteasy-jackson-provider:2.+'
  providedCompile 'javax.servlet:javax.servlet-api:3.+'
}

jettyRun.webAppSourceDirectory = file("src/main/resources")

task seed(type:JavaExec) {
   main = 'ch.issueman.webservice.Seed'
   classpath = sourceSets.main.runtimeClasspath
}

Jetty is our application server, instead of the default webapp folder we will use the resources folder to configure the application. The MySQL connector is the database driver for EclipseLink. The custom seed task allows us to execute the main method of our Seed class.

client/build.gradle

dependencies {
  compile project(':common')
}

The client uses common dependencies only.

Run two gradle tasks on the command line.
- gradle build: This will build the Java application project.
- gradle eclipse: Creates the required files such as .project to import the projects into eclipse.
Finally set the workspace path of eclipse to the folder that contains the <project> directory and import the projects within the <project> folder.

Congratulation you’ve finished the first part of this tutorial. Can’t wait to see you in the next part ;)

Update

2015-03-28: I’ve replaced Hibernate with EclipseLink and dropped FlywayDB as we create the database with EclipseLink.
2015-04-01: Added project lombok to eclipse and and project dependencies.

Links

Here are all links to the other parts of this tutorial.

Source

Several Java tutorials from Mykong.com
How do I use Lombok with Eclipse, Gradle, and Javadocs?

Superb Java development with Gradle, Hibernate, FlywayDB, JavaFX and Eclipse - Part 1

Tue, 03 Mar 2015 23:53:04 +0000

Wow that’s the best Java tutorial I’ve seen so far!

-You in a few minutes.

Sounds selfish, doesn’t it? Well it has the right to be, it took me 2 hours to write that! Don’t hesitate now and turn on your dev machine. After this tutorial you have to rethink your Java development process and if you don’t have one yet this is the best place to start. I will show you the most advanced tools to speed up your Java development environment and collaboration capabilities. In this tutorial we are going to develop a simple MVC (Model, View, Controller) application ruled by the DRY (Don’t repeat yourself) approach.

The crappy picture below shows what’s going on according to my brain.

So what do I excepect from you? Well, make sure you know at least a little bit about the following topics:

Java Collection Framework
Java Classes and Interfaces
Java Development with Eclipse
MySQL database and SQL scripts
Feeling comfortable with the command line

Let’s get startet with the tools. You don’t have to install them yet, first I will explain them in detail.

Gradle: A task runner and package manager for your Java project (in case you’re a web developer that would be bower and grunt).
Hibernate ORM: Object Relation Manager that simplifies database transactions by applying an abstraction layer.
Flyway: An awesome database migration tool (works perfect within a team and git as version control).
JavaFX: Framework for rich application.
Eclipse: The development IDE of your choice.
MySQL server: OpenSource database server.

Got it? Now we are going to install some stuff.

Installation

Start with MySQL. The easiest way to get MySQL on your computer is to install a whole web development stack. They usually come with phpmyadmin which is greate to view your database content and other usefull tools preinstalled. I recommand the Nginx Stack by Bitnami.
Next, if not done yet, install Eclipse.
And finally get Gradle to run.

As told Gradle is our servant for the project. Follow these steps to set up our project directory and dependencies.

Create a project folder somewhere on your disk C:/../<project>.
Open the command line (on windows please use PowerShell) and open the project folder cd C:/../<project>.
In the project folder create a file named build.gradle.
In the order as showed below, paste the snippets into the file.

build.gradle - start

buildscript {
    repositories {
        mavenCentral()
    }
    dependencies {
        classpath 'mysql:mysql-connector-java:5.1.34'
        classpath 'org.flywaydb:flyway-gradle-plugin:3.1'
    }
}

Adds the Flyway database migration tool to your gradle task library.

apply plugin: 'java'
apply plugin: 'eclipse'
apply plugin: 'flyway'

Gradle plugins to set up the project.

repositories {
    mavenCentral()
}

Gradle gets dependencies from the maven repository.

dependencies {
    compile 'org.hibernate:hibernate-core:4.3.8.Final'
    compile 'mysql:mysql-connector-java:5.1.34'
    
    testCompile group: 'junit', name: 'junit', version: '4.+'
}

These commands compile hibernate, mysql and depending libraries for our Java project.

flyway {
    url = 'jdbc:mysql://localhost:3306'
    user = 'issuemanager'
    password = 'issuemanager'
    schemas = ['issuemanager']
}

Flyway task configuration section. Make sure to update these fields.

build.gradle - end

Open http://localhost/phpmyadmin, login and setup a user and a database with the same definitions.
On your command line run: gradle build. This will download and compile all dependencies.
Run: gradle eclipse. This will create the files required to import the project into eclipse.
Start eclipse and import the project. Don’t close the command line yet!
Update the eclipse project structure according to the yellow marked files and directories showed below. Leave all the files except build.gradle empty.

Update the *.sql files with these statements:

V1__create_person_table.sql

create table person(
    id int NOT NULL AUTO_INCREMENT, PRIMARY KEY (id),
    name varchar(100) NOT NULL
);

This snippets creates the MySQL person table.

V2__add_people.sql.sql

insert into PERSON (NAME) values ('Axel');
insert into PERSON (NAME) values ('Mr. Foo');
insert into PERSON (NAME) values ('Ms. Bar');

This snippet adds people to the persons table.

Switch back to your command line and run: gradle flywayMigrate. This will create the person table according to your snippets.

If everything worked fine so far we are ready to write some code.

Coding - Controller

We will start with the DAO interface and the CRUD (create, read, update, delete) controller.

Update the Data Access Object interface.

DAO.java

package ch.hslu.issueman;

import java.io.Serializable;
import java.util.List;

public interface DAO<T, Id extends Serializable> {
	
	public void persist(T t);
	public T getById(Id id);
	public List<T> getAll();
	public void update(T t);
	public void delete(T t);
	public void deleteAll();
}

This interface defines the functions required to communicate with the database later. As you can see we will use generics as we don’t want to create the same interface implementation for every model (despite we only have one model in this example).

Update the basic Controller class.

I will split the controller in multiple snippets and explain what’s going on in detail.

Controller.java - start

package ch.hslu.issueman;

import java.io.Serializable;
import java.util.List;

import org.hibernate.Session;
import org.hibernate.SessionFactory;
import org.hibernate.Transaction;
import org.hibernate.boot.registry.StandardServiceRegistryBuilder;
import org.hibernate.cfg.Configuration;

public class Controller<T, Id extends Serializable> implements DAO<T, Id> {

	private final Class<T> clazz;
	private Session currentSession;
	private Transaction currentTransaction;
	
	protected Controller(Class<T> clazz) {
        this.clazz = clazz;
    }

clazz and the extended Controller constructor is a workaround to store the generic class type and retrieved it later.
currentSession is used to store a MySQL database access session.
currentTransaction is helper to manage database commits.

public Session openCurrentSession() {
		currentSession = getSessionFactory().openSession();
		return currentSession;
	}
	public Session openCurrentSessionwithTransaction() {
		currentSession = getSessionFactory().openSession();
		currentTransaction = currentSession.beginTransaction();
		return currentSession;
	}	
	public void closeCurrentSession() {
		currentSession.close();
	}
	public void closeCurrentSessionwithTransaction() {
		currentTransaction.commit();
		currentSession.close();
	}	
	private static SessionFactory getSessionFactory() {
		Configuration configuration = new Configuration().configure();
		StandardServiceRegistryBuilder builder = new StandardServiceRegistryBuilder()
				.applySettings(configuration.getProperties());
		SessionFactory sessionFactory = configuration.buildSessionFactory(builder.build());
		return sessionFactory;
	}

	public Session getCurrentSession() {
		return currentSession;
	}
	public void setCurrentSession(Session currentSession) {
		this.currentSession = currentSession;
	}
	public Transaction getCurrentTransaction() {
		return currentTransaction;
	}
	public void setCurrentTransaction(Transaction currentTransaction) {
		this.currentTransaction = currentTransaction;
	}

These lines provide methods to interact with the hibernate session manager. Not very interesting also quite common. Below you’ll see that we will use CurrentSession to read from the database and CurrentSessionwithTransaction for operations such as delete in the database.

	public void persist(T t) {
		openCurrentSessionwithTransaction();
		getCurrentSession().save(t);
		closeCurrentSessionwithTransaction();
	}

	public T getById(Id id) {
		openCurrentSession();
		@SuppressWarnings("unchecked")
		T t = (T) getCurrentSession().get(clazz, id);
		closeCurrentSession();
		return t;
	}

	@SuppressWarnings("unchecked")
	public List<T> getAll() {
		openCurrentSession();
		List<T> list = (List<T>) getCurrentSession().createQuery("from " + clazz.getName()).list();
		closeCurrentSession();
		return list;
	}

	public void update(T t) {
		openCurrentSessionwithTransaction();
		getCurrentSession().update(t);
		closeCurrentSessionwithTransaction();
	}

	public void delete(T t) {
		openCurrentSessionwithTransaction();
		getCurrentSession().delete(t);
		closeCurrentSessionwithTransaction();
	}
	
	public void deleteAll() {
		openCurrentSessionwithTransaction();
		List<T> people = getAll();
		for (T t : people) {
			delete(t);
		}
		closeCurrentSessionwithTransaction();
	}

	public void printToJson(List<T> l) {
		
		int size = l.size();
		System.out.println("[");
		for (T i : l) {
			System.out.println(((Model) i).toJson());
			if (--size != 0){System.out.print(",");}
		}
		System.out.print("]");			
	}
}

This is the implementation of the DAO interface. Checkout the getAll method, there you’ll see how the class name workaround is used. The console output of values is Json styled.

Controller.java - end

That’s it, this is our basic Controller that will be used by each Model Controller to interact with the database.

Coding - Model

Now you will see the allmighty power of hibernate. It never was easier to connect a model with a SQL table.

Update the model interface.

Model.java

package ch.hslu.issueman;

public interface Model {
	public String toJson();
}

Easy but usefull. Every model has to implement the toJson method, because why not?

Update the Person model with some special markup.

Person.java

package ch.hslu.issueman;

import javax.persistence.Column;
import javax.persistence.Entity;
import javax.persistence.GeneratedValue;
import javax.persistence.GenerationType;
import javax.persistence.Id;
import javax.persistence.Table;

@Entity
@Table(name="person")
public class Person implements Model{
	
	@Id
	@GeneratedValue(strategy=GenerationType.AUTO)
	@Column(name="id")
    private int id; 
	
	@Column(name="name")
    private String name;
	
	public Person() {}

	public Person(String name) {
		setName(name);
	}
	
	public int getId() {
		return id;
	}
		
	public String getName() {
		return name;
	}
	
	public void setName(String name) {
		this.name = name;
	}
	
	public String toJson() {
		return "{\"id\": \"" + getId() + "\", \"name\":\"" + getName() + "\"}";
	}

}

Those @-Annotation will tell hibbernate which column belongs to which object attribute. Awesome! Isn’t it?

Update the hibernate configuration file.

hibernate.cfg.xml

<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE hibernate-configuration SYSTEM 
"http://www.hibernate.org/dtd/hibernate-configuration-3.0.dtd">

<hibernate-configuration>
   <session-factory>
   <property name="hibernate.dialect">org.hibernate.dialect.MySQLDialect</property>
   <property name="hibernate.connection.driver_class">com.mysql.jdbc.Driver</property>
   <property name="hibernate.connection.url">jdbc:mysql://localhost/issuemanager</property>
   <property name="hibernate.connection.username">issuemanager</property>
   <property name="hibernate.connection.password">issuemanager</property>
   <property name="show_sql">true</property>
   <property name="hibernate.current_session_context_class">thread</property>
   <mapping class="ch.hslu.issueman.Person"/>
</session-factory>
</hibernate-configuration>

Add the same values as already used to configure flyway. Very important is the mapping property which finalizes the mapping process of the model with database table.

Coding - App

This is the last coding part of the first tutorial.

Update the Person Controller.

PersonController.java

package ch.hslu.issueman;

import java.util.List;

public class PersonController implements DAO<Person, Integer>{
	
	private static Controller<Person, Integer> controller;
	
	public PersonController() {
		controller = new Controller<Person, Integer>(Person.class);
	}
	
	public void persist(Person person) {
		controller.persist(person);
	}

	public Person getById(Integer id) {
		Person person = controller.getById(id);
		return person;
	}

	public List<Person> getAll() {
		List<Person> people = controller.getAll();
		return people;
	}

	public void update(Person person) {
		controller.update(person);
	}

	public void deleteById(Integer id) {
		controller.delete(controller.getById(id));
	}

	public void delete(Person person) {
		controller.delete(person);
	}
	
	public void deleteAll() {
		controller.deleteAll();
	}

	public void printToJson(List<Person> people) {
		controller.printToJson(people);	
	}
}

As you can see the Person Controller uses the basic Controller to do some CRUD actions. Every model controller must implement the DAO interface and must use the basic Controller. This strategy allows us to define model depended actions within the CRUD methods.

Update the application code.

App.java

package ch.hslu.issueman;

public class App {

	public static void main(String[] args) {
		
		PersonController PersonController = new PersonController();
//		PersonController.deleteAll();
		
//		Person person4 = new Person("Fyodor Dostoevsky");
//		Person person5 = new Person("Leo Tolstoy");
//		Person person6 = new Person("Jane Austen");
//		
//		PersonController.persist(person4);
//		PersonController.persist(person5);
//		PersonController.persist(person6);

//		PersonController.deleteById(6);
		
//		person4.setName("Frodo");
//		PersonController.update(person4);
				
		PersonController.printToJson(PersonController.getAll());
	
		System.exit(0);
	}

}

This example application class shows how the Person Controller is used to retrieve and store Person object. Uncomment or comment sections to test the CRUD methods.

Finally run the application.

You should see a lot of hibernate blabla and of course the Json outpout of the person table.

Hibernate: select person0_.id as id1_0_, person0_.name as name2_0_ from person person0_
[
{"id": "1", "name":"Axel"}
,{"id": "2", "name":"Mr. Foo"}
,{"id": "3", "name":"Ms. Bar"}
]

That was the first part of this tutorial and of course it was just a peek into these great tools. Hibernate offers much more possibilites to connect different models and do other awesome stuff. So don’t miss the documentation on their website or other tutorials. I hope to see you soon in the next chapter when we create a rich client application with JavaFX.

Update

2015-03-05: Part 2 has been released: https://janikvonrotz.ch/2015/03/05/superb-java-development-with-gradle-hibernate-flywaydb-javafx-and-eclipse-part-2/

Source

You can get the project from here: https://codeberg.org/janikvonrotz/issue-manager/releases/tag/v1.0

Unattended Encrypted Incremental Backup to Amazon S3

Fri, 25 Apr 2014 07:28:09 +0000

This post is part of my Your own Virtual Private Server hosting solution project.
Get the latest version of this article here: https://gist.github.com/9410478.

Introduction

For this task we are going to configure a duplicity script wrapper. Unregarded of the installation instructions it’s expected that you have already signed up for an Amazon account and know how to use their services.

Requirements

Installation

Clone the GitHub project.

cd /usr/local/src
sudo git clone https://github.com/zertrin/duplicity-backup.git

Copy the configuration file.

cd /usr/local/src/duplicity-backup
sudo mkdir /etc/duplicity-backup
sudo cp duplicity-backup.conf.example /etc/duplicity-backup/duplicity-backup.conf

Edit the configuration file.

sudo vi /etc/duplicity-backup/duplicity-backup.conf

Update the AWS credentials.

AWS_ACCESS_KEY_ID="[aws key id]"
AWS_SECRET_ACCESS_KEY="[aws access key]"

Update the gpg encryption settings.

PASSPHRASE="[gpg passphrase]"
GPG_ENC_KEY="[gpg key id]"
GPG_SIGN_KEY="[gpg key id]"

In case you can’t remember the gpg key id use the GnuPG tool.

gpg -k

Where the gpg key id is displayed in the line pub 2048R/>>C58886FB<< 2014-03-14

Set the backup start directory.

ROOT="/var"

Set the S3 destination bucket.

DEST="s3+http://[bucket name]/[backup-folder]/"

Define which folder should be included in the backup.

INCLIST=(  "/var/backups/mysql/latest" 
           "/var/www/<wordpress>/wp-content" 
        )

Define which folders inside the include folders should be ignored.

EXCLIST=(  "/var/www/[wordpress]/wp-content/backupwordpress*" )

Update the logging settings.

LOGDIR="/var/log/duplicity-backup/"
LOG_FILE="duplicity-`date +%Y-%m-%d_%H-%M`.txt"
LOG_FILE_OWNER="[group]:[user]"

Don’t forget to create the log folder.

sudo mkdir /var/log/duplicity-backup

Run the script to check wether it works or not.

sudo /usr/local/src/duplicity-backup/duplicity-backup.sh -c /etc/duplicity-backup/duplicity-backup.conf -b

To debug errors you can add the parameter -d, adjust the verbosity level with -v[1-9] and check the duplicity log.

In case you’ll get the error Import of duplicity.backends.giobackend Failed: No module named gio or BackendException: Could not initialize backend: No module named paramiko you have to install some additional pyhton packages.

sudo apt-get install python-paramiko python-gobject-2

In addition if you want to use gdocs as a destination you have to install the according python libarary.

sudo apt-get install pyhton-gdata

You can list the current backup with the parameter --list-current-files.

sudo /usr/local/src/duplicity-backup/duplicity-backup.sh -c /etc/duplicity-backup/duplicity-backup.conf --list-current-files

Get further help for the backup script by running the script without parameters.

 sudo /usr/local/src/duplicity-backup/duplicity-backup.sh

scheduling the job is easily done by adding a new line to the cron configuration file.

sudo vi /etc/crontab

Add the backup schedule command. The following example is executed daily a 7 o’clock.

00 7    * * *   [user]   sudo /usr/local/src/duplicity-backup/duplicity-backup.sh -c /etc/duplicity-backup/duplicity-backup.conf -b

Watch out for the schedule time of the automysqlbackup, you should schedule the duplicity backup job after the automysqlbackup job is done.

Finally backup your configurations.

cd ~
sudo /usr/local/src/duplicity-backup/duplicity-backup.sh -c /etc/duplicity-backup/duplicity-backup.conf --backup-script

Answert the prompt as showed bleow.

>> Are you sure you want to do that ('yes' to continue)?
yes

Enter passphrase: [gpg passphrase]

Source

Automatic Amazon s3 Backups on Ubuntu / Debian
GitHub duplicity-backup by zertrin
GitHub issue: No module named gio
Missing modules for paramiko and gio in duplicity foo

Migrate Piwik website

Thu, 24 Apr 2014 07:23:42 +0000

This post is part of my Your own Virtual Private Server hosting solution project.
Get the latest version of this article here: https://gist.github.com/9640572.

Introduction

This article assumes you’re going to move an exisiting piwik website from one server to another.

Requirements

Instructions

Export the existing SQL Piwik database with phpMyAdmin.

Import the database export into the new Piwik database with phpMyAdmin.

Update the tracking codes or update Piwik urls on the Piwik tracked websites if the hosting has changed.

Install piwik website

Tue, 22 Apr 2014 06:43:16 +0000

This post is part of my Your own Virtual Private Server hosting solution project.
Get the latest version of this article here: https://gist.github.com/9640540.

Introduction

Piwik is the leading open source web analytics platform that gives you valuable insights into your website’s visitors, your marketing campaigns and much more, so you can optimize your strategy and online experience of your visitors.

Requirements

Installation

Create the new Piwik website folder.

sudo mkdir /var/www/[piwik]
cd /var/www/[piwik]

Download latest piwik

sudo wget http://builds.piwik.org/latest.tar.gz
sudo tar -xzvf latest.tar.gz

Copy Piwik content and delete unnecessary files.

sudo cp -r ./piwik/* ./
sudo rm -r piwik
sudo rm latest.tar.gz "How to install Piwik.html"

Let’s create the MySQL Piwik database and user.

mysql -u root -p

Enter the MySQL root user password.

Create the Piwik database.

CREATE DATABASE [piwik];

Create the Piwik database user.

CREATE USER [piwik]@localhost;

Set the password for the Piwik database user.

SET PASSWORD FOR [piwik]@localhost = PASSWORD("[password]");

Grant Piwik user full access on Piwik database.

GRANT ALL PRIVILEGES ON [piwik].* TO [piwik]@localhost IDENTIFIED BY '[password]';

Refresh MySQL and exit.

FLUSH PRIVILEGES;
exit

Add the Nginx configuration to an existing website.

server{
    
    ...
    
    location /piwik{
        root /var/www;
    }
    
    ...
    
    location ~ .php$ {
        
        ...
        
        if ($request_uri ~* /piwik) {
            set $php_root /var/www;
        }
        
        ...
    }
}

Provide access to the piwik folder.

sudo chown -R www-data:www-data /var/www/piwik

Finally let’s add the archive cron job which will highly improve the processing time for your piwik reports.

Add a new cron job.

sudo vi /etc/cron.d/piwik-archive

Add this content to the cron file.

MAILTO="[mail@example.com]"
5 * * * * www-data /usr/bin/php5 /var/www/[piwik]/console core:archive --url=http://[host]/piwik/ > /var/log/piwik/archive.log

Then create the log folder and grant access for the user.

sudo mkdir /var/log/piwik
sudo chown www-data:www-data piwik

Test config and reload Nginx service.

sudo nginx -t && sudo service nginx reload

Open your browser on //[host]/piwik and install the Piwik website.

Source

How to Install Piwik on Ubuntu by AdminEmpire

Install phpMyAdmin website

Mon, 14 Apr 2014 09:49:37 +0000

This post is part of my Your own Virtual Private Server hosting solution project.
Get the latest version of this article here: https://gist.github.com/9392925.

Introduction

phpMyAdmin is a free software tool written in PHP, intended to handle the administration of MySQL over the Web.

Requirements

Installation

Start the installation phpMyAdmin.

sudo apt-get install phpmyadmin

As we use nginx for this installation, hit Tab and Enter on the first prompt.

Chose and enter the MySQL root password on the second prompt.

Create a secure password for phpMyAdmin. Don’t use the MySQL root password!

Add the phpMyAdmin Nginx configuration to one of your websites.

server{
    ...
    
    root /usr/share;

    ...
    
    location ~ .php$ {
        ...
        
        # change the php root for phpMyAdmin
        if ($request_uri ~* /phpmyadmin) {
            set $php_root /usr/share;
        }
        
        ...
    }
}

Test config and reload Nginx service.

sudo nginx -t && sudo service nginx reload

Open your browser on //[host]/phpmyadmin.

Install automysqlbackup

Tue, 08 Apr 2014 08:33:11 +0000

This post is part of my Your own Virtual Private Server hosting solution project.
Get the latest version of this article here: https://gist.github.com/9488132.

Introduction

AutoMySQLBackup with a basic configuration will create Daily, Weekly and Monthly backups of one or more of your MySQL databases from one or more of your MySQL servers.

Requirements

Installation

First create the backup folder.

sudo mkdir /var/backups/mysql

Install the latest version of automysqlbackup from the official website.

sudo mkdir /usr/local/src/automysqlbackup
cd /usr/local/src/automysqlbackup
sudo wget http://switch.dl.sourceforge.net/project/automysqlbackup/AutoMySQLBackup/AutoMySQLBackup%20VER%203.0/automysqlbackup-v3.0_rc6.tar.gz
sudo tar xvzf automysqlbackup-v3.0_rc6.tar.gz
sudo ./install.sh

Answer the install wizard as showed below.

global configuration diectory: default
directory for the executable: default

Add read permissios for all in the config folder.

cd /etc/automysqlbackup
sudo chmod a+r ./*

Update the automysqlbackup config file.

sudo vi /etc/automysqlbackup/automysqlbackup.conf

Set credentials and update the backup path.

# Username to access the MySQL server e.g. dbuser
CONFIG_mysql_dump_username='root'

# Password to access the MySQL server e.g. password
CONFIG_mysql_dump_password='[password]'

# Backup directory location e.g /backups
CONFIG_backup_dir='/var/backups/mysql'

Unlock the latest backup in order to make backups for third party programs available

# Store an additional copy of the latest backup to a standard
# location so it can be downloaded by third party scripts.
CONFIG_mysql_dump_latest='yes'

Let’s schedule the backup job by adding a daily cron script.

sudo vi /etc/cron.daily/automysqlbackup

With the following content.

#!/bin/sh

/usr/local/bin/automysqlbackup /etc/automysqlbackup/automysqlbackup.conf

chown root.root /var/backup/mysql* -R
sudo chmod -R a-x+X /var/backup/mysql

Enable execution of the script for the owner.

sudo chmod o+x /etc/cron.daily/automysqlbackup

You can run the backup script manually.

cd /etc/cron.daily
sudo ./automysqlbackup

Checkout the backup folder.

cd /var//backup/mysql

Source

How To Backup MySQL Databases on an Ubuntu VPS by Digital Ocean
How to back up MySQL databases using AutoMySQLBackup by A2 hosting automysqlbackup website

Install MySQL

Mon, 07 Apr 2014 07:14:46 +0000

This post is part of my Your own Virtual Private Server hosting solution project.
Get the latest version of this article here: https://gist.github.com/9392658.

Introduction

MySQL is the world’s most popular open source database system.

Requirements

Ubuntu server

Installation

Install MySQL server

sudo apt-get install mysql-server

Set the mysql root user password during the installation

Install the default MySQL databases

sudo mysql_install_db

Run the finisher script and respond except for the first prompt with yes in order to get a secure MySQL installation

sudo /usr/bin/mysql_secure_installation

Connect to your new MySQL server

mysql -uroot -p

Enter the root password

And run this command to get the MySQL version

SHOW variables LIKE "%version%";

Source

Ubuntu MySQL server guide

Step by Step: Install WordPress Blog

Sat, 22 Mar 2014 10:26:15 +0000

Latest Version of this guide: https://gist.github.com/9320678

Finishing this guide you’ll get:

A running WordPress installation
Nginx proxy with PHP and Fast CGI
MySQL server accessible with phpMyAdmin

Specification of latest running installation:

Date: 03.03.2014
OS: Ubuntu 64 bit - 12.04.4 LTS
Provider: Amazon EC2
Browser: Google Chrome - 33.0.1750.117
WordPress: 3.8.1
Nginx: 1.1.19
MySQL: 5.5.35
PHP: 5.3.10
phpMyAdmin: 3.4.10.1

Requirements

Server is behind a firewall, that only allows http, https and ssh
The server is accessed with ssh keys (user password authentication must be disabled)
Server is not accessed with the root user
You’re able to edit files with VI

Ubuntu

Update Ubuntu

sudo apt-get update && sudo apt-get upgrade

Install additional packages

sudo aptitude install build-essential zip git

MySQL

Install MySQL server and php5 MySQL module

sudo apt-get install mysql-server php5-mysql

Set the mysql root user password during the installation

Install the default MySQL databases

sudo mysql_install_db

Run the finisher script and respond every prompt with yes to get a secure MySQL installation

sudo /usr/bin/mysql_secure_installation

Connect to your new MySQL server

mysql -uroot -p

Enter the root password

And run this command to get the MySQL version

SHOW variables LIKE "%version%";

Nginx

Install Nginx

sudo apt-get install nginx

Create a Nginx site configuration file

sudo touch /etc/nginx/sites-available/wordpress.conf
sudo vi /etc/nginx/sites-available/wordpress.conf

Paste this config

server {    
listen   80;

	root /var/www/wordpress;
	index index.php index.html index.htm;

	server_name [example.com];

	location / {
		try_files $uri $uri/ /index.php?q=$uri&$args;
	}

	error_page 404 /404.html;

	error_page 500 502 503 504 /50x.html;
	location = /50x.html {
		root /usr/share/nginx/www;
	}
    
    client_max_body_size 10M;

	# pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
	location ~ \.php$ {
		try_files $uri = 404;
		#fastcgi_pass 127.0.0.1:9000;
		# With php5-fpm:
		fastcgi_pass unix:/var/run/php5-fpm.sock;
		fastcgi_index index.php;
		include fastcgi_params;
	}
}

Optionally you can rewrite false urls to a specified canonical url

server {
    listen       80;
    server_name  www.example.com  example.com;
    if ($http_host = www.example.org) {
        rewrite  (.*)  http://[example.com]$1;
    }
    ...
}

Create a symlink to the config file

sudo ln -s /etc/nginx/sites-available/wordpress.conf /etc/nginx/sites-enabled/wordpress.conf

Restart Ngnix

sudo service nginx restart

Check Nginx version

nginx -v

PHP

Install PHP with FastCGI support

sudo apt-get install php5-fpm

configure PHP installaton

 sudo vi /etc/php5/fpm/php.ini

Find the line cgi.fix_pathinfo=1 by pressing ESC and enter

/;cgi.fix_pathinfo=1

Uncomment this line and change value to 0

cgi.fix_pathinfo=0

If this number is kept as 1, the php interpreter will do its best to process the file that is as near to the requested file as possible. This is a possible security risk. If this number is set to 0, conversely, the interpreter will only process the exact file path—a much safer alternative.

Find the line ; default extension directory. and insert below

extension=mcrypt.so

Update the listening port for the php fpm

sudo vi /etc/php5/fpm/pool.d/www.conf
Set listen = /var/run/php5-fpm.sock

Restart the service

sudo service php5-fpm restart

Create the website folder

sudo mkdir /var/www/wordpress

Add a PHP info file

sudo vi /var/www/wordpress/info.php

Set content

<?php
phpinfo();
?>

Open your browser on http://example.com/info.php

Delete this file if everything works

phpMyAdmin

Install phpMyAdmin

sudo apt-get install phpmyadmin

When phpMyAdmin prompts you to choose a server (either apache or lighttpd)hit tab, and select neither one.

When phpMyAdmin asks you wether to configure database for phpmyadmin with dbconfig-common. Chose and enter the MySQL root user password

Hit on the MySQL application password for phpmyadmin prompt.

Create a symbolic link for the phpMyAdmin website

sudo ln -s /usr/share/phpmyadmin/ /usr/share/nginx/www

Create a Nginx configuration file

sudo touch /etc/nginx/sites-available/phpmyadmin.conf
sudo vi /etc/nginx/sites-available/phpmyadmin.conf

Paste this config

server{
	listen 80;
	
	server_name [Your Public IP];
	root /var/www/;
	
	index index.php index.html index.htm;
	
    client_max_body_size 10M;
    
	location ~ .php$ {
		try_files $uri = 404;
		fastcgi_pass unix:/var/run/php5-fpm.sock;
		fastcgi_index index.php;
		include /etc/nginx/fastcgi_params;
	}
}

Create a symlink to the config file

sudo ln -s /etc/nginx/sites-available/phpmyadmin.conf /etc/nginx/sites-enabled/phpmyadmin.conf

Restart Ngnix

sudo service nginx restart

Open the browser on http://[YourPublicIP]/phpmyadmin/

WordPress

Open the WordPress site directory

cd /var/www/wordpress/

Download latest WordPress package and untar it

sudo wget http://wordpress.org/latest.tar.gz
tar -xzvf latest.tar.gz

Copy the untared files to the current folder and delete the other files

sudo cp -r ./wordpress/* ./
sudo rm -r wordpress
sudo rm latest.tar.gz

Let’s create the MySQL WordPress user

mysql -u root -p

Enter the MySQL root user password

Create the WordPress database

CREATE DATABASE wordpress;

Create the WordPress database user

CREATE USER wordpress@localhost;

Set the password for the WordPress database user

SET PASSWORD FOR wordpress@localhost = PASSWORD("[password]");

Grant WordPress user full access on WordPress database

GRANT ALL PRIVILEGES ON wordpress.* TO wordpress@localhost IDENTIFIED BY '[password]';

Refresh MySQL and exit

FLUSH PRIVILEGES;
exit

Copy the WordPress example config file

sudo cp wp-config-sample.php wp-config.php

Edit the config file

sudo vi wp-config.php

Set database, database user and his password

define('DB_NAME', 'wordpress');

define('DB_USER', 'wordpress');

define('DB_PASSWORD', '[password]');

Update permissions for Nginx user

sudo chown www-data:www-data * -R 
sudo usermod -a -G www-data www-data

Open the browser on http://example.com and install you WordPress blog

Source

Install MySQL
How To Install Linux, nginx, MySQL, PHP (LEMP) stack on Ubuntu 12.04 by Digital Ocean
How To Install phpMyAdmin on a LEMP server by Digi
How To Install Wordpress with nginx on Ubuntu 12.04 by Digital Ocean
Nginx rewrite rules
Get MySQL version