Scripts et taches planifiees » Historique » Version 72
Version 71 (Julien Enselme, 06/08/2013 13:08) → Version 72/118 (Julien Enselme, 06/08/2013 13:10)
Afin de gagner du temps et d'éviter les erreurs humaines, des scripts ont été écrits tout au long du projet. Certains sont même exécutés automatiquement toutes les semaines.
{{>toc}}
h1. Les tâches planifiées
h2. Crontab
Pour exécuter ces tâches régulièrement, on utilise le "crontab":http://fr.wikipedia.org/wiki/Crontab. Il s'agit d'un programme installé sur notre serveur.
Pour voir et modifier la liste des actions :
# se connecter au serveur : <code>ssh assos@sas1.centrale-marseille</code>
# taper la commande pour voir le crontab <code>crontab -l</code>
# taper la commande pour modifier le crontab <code>crontab -e</code>.
{{important(Ne pas effectuer cette opération avant de s'être renseigné sur "vi":http://fr.wikipedia.org/wiki/Vi (l'éditeur de texte utilisé qui n'est pas vraiment intuitif :p) et sur "la syntaxe du crontab":http://fr.wikipedia.org/wiki/Crontab#Modification !
Les scripts doivent si possible s’exécuter avant le club Drupal pour que les problèmes éventuels y soient réglés)}}
{{tip(
* Pour utiliser un autre éditeur que vi taper : export EDITOR=editor_name exemple : vim, nano.
* d7-all-update-contrib.sh fait des sauvegardes automatiques individuelles pour les sites drupal)}}
<pre>
<code class="php">
######## How to use
# mm hh jj MMM JJJ task
# minutes hour day_month month day_week
# for more informations : https://fr.wikipedia.org/wiki/Crontab#Notation
######## End how to
### Cleaning
00 5 * * * /users/guest/assos/bin/all-reset-permissions.sh > /dev/null 2>&1
00 6 * * * /users/guest/assos/bin/delete-temporary-files.sh > /dev/null 2>&1
### Traqu-heure
00 7 * * * /users/guest/assos/bin/traqu_heure/script-restitution.sh > /dev/null 2>&1
0 * * * * /users/guest/assos/bin/traqu_heure/script-traque.sh > /dev/null 2>&1
### Databases
# Create backup
00 8 1,15 * * /users/guest/assos/bin/d7-all-dump-full.sh > /dev/null 2>&1
00 1 * * 2 /users/guest/assos/bin/d6-all-dump-full.sh > /dev/null 2>&1
# flush backup
00 8 2,16 * * /users/guest/assos/bin/flush-full-databases-backups.sh d7 > /dev/null 2>&1
00 22 * * 2 /users/guest/assos/bin/flush-full-databases-backups.sh d6 > /dev/null 2>&1
00 23 * * 2 /users/guest/assos/bin/flush-individual-databases-backups.sh auto d7 > /dev/null 2>&1
00 1 * * 3 /users/guest/assos/bin/flush-individual-databases-backups.sh manual d7 > /dev/null 2>&1
### Drupal 7
00 22 * * 3 /users/guest/assos/bin/d7-all-update-contrib.sh | mail -s "d7-all-update-contrib.sh" assos
00 1 * * 4 /users/guest/assos/bin/d7-all-update-localisation.sh | mail -s "d7-all-update-localisation.sh" assos
00 2 * * 4 /users/guest/assos/bin/d7-all-security-review.sh
00 3 * * 4 /users/guest/assos/bin/d7-all-reset-variables.sh > /dev/null 2>&1
00 4 * * 4 /users/guest/assos/bin/d7-all-releases-info.sh | mail -s "d7-all-releases-info.sh" assos
### Drupal 6
00 22 * * 6 /users/guest/assos/bin/d6-all-update-contrib.sh > /dev/null 2>&1
00 23 * * 6 /users/guest/assos/bin/d6-all-reset-variables.sh > /dev/null 2>&1
# Launch drush cron on fellowings
45 17 * * * /users/guest/assos/bin/drushcronone assos.centrale-marseille.fr.ginfo > /dev/null 2>&1
*/15 * * * * /users/guest/assos/bin/drushcronone assos.centrale-marseille.fr.clubfinance > /dev/null 2>&1
45 3 * * * /users/guest/assos/bin/drushcronone default > /dev/null 2>&1
35 00 * * * /users/guest/assos/bin/drushcronone assos.centrale-marseille.fr.cac13 > /dev/null 2>&1
*/45 * * * * /users/guest/assos/bin/drushcronone assos.centrale-marseille.fr.expat > /dev/null 2>&1
*/10 * * * * /users/guest/assos/bin/drushcronone forum.centrale-marseille.fr > /dev/null 2>&1
00 22 * * 7 /users/guest/assos/bin/disk-usage.sh | mail -s "disk-usage.sh" assos
</code>
</pre>
La liste des tâches effectuées régulièrement est ci-dessous.
h2. La réinitialisation des variables
Cette tâche consiste à réinitialiser certaines variables qui donnent des droits considérés comme trop permissifs donc dangereux aux administrateurs des sites ou nécessaire pour d'autres scripts tels que piwik ou security_review.
En voici quelques exemples :
<pre>
<code class="bash">
drush -y vset error_level 0
</code>
</pre>
Cette commande permet de ne pas afficher les messages d'erreurs. En effet, ils contiennent parfois des informations sensibles sur l'installation et ne doivent donc pas être divulgués à n'importe qui.
<pre>
<code class="php">
drush -y vset --always-set reverse_proxy TRUE
drush -y vset --always-set --format=json reverse_proxy_addresses '["147.94.19.16","147.94.19.17"]'
</code>
</pre>
Ces commandes permettent de déclarer à Drupal les serveurs proxy du CRI afin d'éviter qu'il ne répertorie tous les visiteurs comme ayant l'adresse des serveurs sus-cités. **Pas fini : voir http://assos.centrale-marseille.fr/lessive/content/t%C3%A2che/rendre-les-settingsphp-conformes-%C3%A0-la-version-712-de-drupal**
h2. La sauvegarde des bases de données
Sur les installations d6 et d7 : une fois par semaine
On utilise les [[Scripts_et_taches_planifiees#d6-all-dump-fullsh|d6-all-dump-full.sh]], [[Scripts_et_taches_planifiees#d6-dump-individualsh|d6-dump-individual.sh]], [[Scripts_et_taches_planifiees#d7-all-dump-fullsh|d7-all-dump-full.sh]] et [[Scripts_et_taches_planifiees#d7-all-dump-individualsh|d7-all-dump-individual.sh]].
h2. La purge des sauvegardes de bases de données bdd
Sur les installations d6 et d7 : toutes les semaines
Cette tâche utilise les scripts le script de purges [[Scripts_et_taches_planifiees#purge_des_sauvegardes.sh|purge des sauvegardes ([[Scripts_et_taches_planifiees#flush-full-databases-backupssh|flush-full-databases-backups.sh]] et [[Scripts_et_taches_planifiees#flush-individual-databases-backupssh|flush-individual-databases-backups.sh]]) sauvegardes]] afin de libérer de l'espace disque en supprimant les sauvegardes de bdd les plus vieilles.
h1. Liste des scripts à disposition
Les scripts utilisés sont hébergés dans le répertoire bin du compte assos.
h2. Configuration et utilitaires
h3. scripts-config.sh
<pre>
<code class="php">
#!/bin/sh
PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/games:/usr/local/sbin:/usr/local/bin:/users/guest/assos/bin
PATH=$PATH:/usr/guest/assos/bin
scripts_config='imported'
email_multi_assos="assos@centrale-marseille.fr"
dir_multi_assos='/users/guest/assos'
dir_scripts="$dir_multi_assos/bin"
dir_template="$dir_multi_assos/template"
###### Database
db_server="myassos.serv.int"
db_user="assos"
dir_log="$dir_multi_assos/log"
###### Backup
db_full_backup_number=3
db_individual_manual_backup_number=2
db_individual_auto_backup_number=3
dir_backup="$dir_multi_assos/backup"
dir_full_backup="$dir_backup/full"
dir_individual_backup="$dir_backup/individual"
dir_individual_manual_backup="$dir_individual_backup/manual"
dir_individual_auto_backup="$dir_individual_backup/auto"
###### D7 variables
d7_dir="$dir_multi_assos/htmltest"
d7_dir_sites="$d7_dir/sites"
### Log
d7_dir_log="$dir_log/d7"
### Backup
d7_dir_full_backup="$dir_full_backup/d7"
d7_dir_individual_manual_backup="$dir_individual_manual_backup/d7"
d7_dir_individual_auto_backup="$dir_individual_auto_backup/d7"
### Template
d7_settings_template_name="settings-D7-bddinde-template.php"
d7_settings_template="$dir_template/$d7_settings_template_name"
###### D6 variables
d6_dir="$dir_multi_assos/html"
d6_dir_sites="$d6_dir/sites"
### Backup
d6_dir_full_backup="$dir_full_backup/d6"
d6_dir_individual_manual_backup="$dir_individual_manual_backup/d6"
d6_dir_individual_auto_backup="$dir_individual_auto_backup/d6"
</code>
</pre>
h3. scripts-config-site.sh
<pre>
<code class="php">
#!/bin/sh
# Sets useful variables of a site.
# ARGS: site_name
# Requires scripts-config.sh
# Check if scripts-config.sh is imported.
if [ -z $scripts_config ] ; then
echo "Import of scripts-config.sh required."
exit 1
fi
if [ -z $1 ] ; then
echo "This script needs a site name as argument."
exit 1
fi
scripts_config_site='imported'
d7_site_name=$1
d7_site_dir="$d7_dir_sites/assos.centrale-marseille.fr.$d7_site_name"
d7_site_settings="$d7_site_dir/settings.php"
</code>
</pre>
h3. scripts-utils.sh
<pre>
<code class="php">
#!/bin/sh
# This script contains useful functions for other scripts.
# Check if scripts-config.sh is imported.
if [ -z $scripts_config ] ; then
echo "Import of scripts-config.sh required."
. scripts-config.sh
fi
scripts_utils='imported'
ask_password() {
# read -s doesn't work with sh.
# usage: pass=`ask_password "password please:"`
echo $1 >&2
echo -n ">" >&2
stty_avant=`stty -g`
stty -echo
read password
stty $stty_avant
echo "$password"
unset password
}
ask_password_db() {
# ARGS: server_name, user_name
local db_password="pour_boucler"
# empty db request to validate password
while ! mysql -h $1 -u $2 -p$db_password -e "" 2>/dev/null ; do
db_password=`ask_password "database password:"`
done
echo $db_password
}
generate_password(){
# ARGS: [password_length]
# The password contains special characters. '/' must be excluded to avoid sed malfunction.
local site_password='/'
if [ -z $1 ] ; then
local password_length=20
else
local password_length=$1
fi
while echo "$site_password" | grep -Fq '/' ; do
site_password=`dd if=/dev/urandom count=1 | uuencode -m - | head -n 2 | tail -n 1 | cut -c-$password_length`
done
echo $site_password
}
count_d7_sites(){
find $d7_dir_sites -type d ! -name all -maxdepth 1 | wc -l
}
check_arguments() {
# ARGS: number of arguments passed to script, number of arguments required, [help text]
if [ $1 -lt $2 ] ; then
echo "Number of arguments insuffisant."
echo $3
exit 1
fi
}
</code>
</pre>
h2. Scripts généraux
h3. all-reset-permissions.sh
<pre>
<code class="php">
#!/bin/sh
. scripts-config.sh
# This script puts the correct permissions to sites folders, settings.php and scripts.
######### drupal 6
for dir in `find $d6_dir_sites -type d -maxdepth 1 -mindepth 1 ! -name all ! -name images ! -name languages` ; do
chmod 755 $dir
chmod 400 $dir/settings.php
done
######### drupal 7
for dir in `find $d7_dir_sites -type d -maxdepth 1 -mindepth 1 ! -name all` ; do
chmod 755 $dir
chmod 400 $dir/settings.php
done
####### bin
chmod -R 700 $dir_scripts
####### backup
chmod -R 700 $dir_backup
####### log
chmod -R 700 $dir_log
</code>
</pre>
h3. contrib-usage.sh
<pre>
<code class="php">
#!/bin/sh
. scripts-config.sh
. scripts-utils.sh
help="# ARGS: Drupal_version, project_status, project_name\n
# List sites that have the project_name with the corresponding project_status.\n
# project status: enabled or disabled"
check_arguments $# 3 "$help"
if [ $1 = d6 ] ; then
cd $d6_dir_sites
else
cd $d7_dir_sites
fi
for dir in `find . -maxdepth 1 -mindepth 1 -type d ! -name all ! -name languages ! -name images | sort ` ; do
# List non-core projects that correspond to the status.
# Keep project_name if listed.
# Count line result. 0 if not listed or 1 if listed.
# Print site_dir if listed.
cd $dir;
if [ 1 -eq `drush pml --no-core --status=$2 | grep $3 | wc -l` ] ; then
echo $dir;
fi
cd -
done
</code>
</pre>
h3. delete-temporary-files.sh
<pre>
<code class="php">
#!/bin/sh
. scripts-config.sh
find $dir_multi_assos -name '*~' -delete
</code>
</pre>
h3. disk-usage.sh
<pre>
<code class="php">
#!/bin/sh
. scripts-config.sh
echo -e "Quotas: 9.1 GB\n"
echo "Assos:"
du -hcs $dir_multi_assos | grep total
echo -e "\n"
echo "D6:"
cd $d6_dir_sites
du -h -d 1
echo -e "\n"
echo "D7:"
cd $d7_dir_sites
du -h -d 1
echo -e "\n"
echo "Logs:"
du -hcs $dir_log | grep total
echo -e "\n"
echo "Backups:"
cd $dir_backup
du -h -d 2
</code>
</pre>
h3. drushcronone
<pre>
<code class="php">
#!/bin/sh
. scripts-config.sh
if [ $# -lt 1 ]; then
echo "usage: $0 <site's folder name>"
exit 1
fi
cd "$d6_sites_dir/$*"
pwd;
drush cron;
</code>
</pre>
h3. flush-files.sh
<pre>
<code class="php">
#!/bin/sh
help="# ARGS: number of files to keep, [email address]\n
# usage: flush files from a folder except a number equal to argument."
# Check if scripts-config.sh is imported.
if [ -z $scripts_utils ] ; then
echo "Import of scripts-utils.sh required."
. scripts-utils.sh
fi
check_arguments $# 1 "$help"
backups_number=`ls | wc -l`
let "number_of_backups_to_delete = backups_number - $1" > /dev/null
if [ $number_of_backups_to_delete -gt 0 ] ; then
ls | head -$number_of_backups_to_delete | xargs rm
else
if [ ! -z $2 ] ; then
dir=`pwd`
echo "There are not enough files in $dir to Flush it. Check if backup script works fine." | mail -s "[db] $dir has a backup problem" $2
fi
fi
</code>
</pre>
h3. flush-full-databases-backups.sh
<pre>
<code class="php">
#!/bin/sh
. scripts-config.sh
# ARGS: Drupal version
cd $dir_full_backup/$1
for dir in `ls` ; do
cd $dir
flush-files.sh $db_full_backup_number $email_multi_assos
cd -
done
</code>
</pre>
h3. flush-individual-databases-backups.sh
<pre>
<code class="php">
#!/bin/sh
. scripts-config.sh
# ARGS: manual or auto, Drupal version
cd $dir_individual_backup/$1/$2
for dir in `ls` ; do
cd $dir
if [ $1 = 'auto' ] ; then
flush-files.sh $db_individual_auto_backup_number $email_multi_assos
else
flush-files.sh $db_individual_manual_backup_number
fi
cd -
done
</code>
</pre>
h2. Scripts Drupal 6
h3. d6-all-drush.sh
<pre>
<code class="php">
#!/bin/sh
. scripts-config.sh
. scripts-utils.sh
help="# ARGS: drush_command"
check_arguments $# 1 "$help"
for dir in `find $d6_dir_sites -maxdepth 1 -mindepth 1 -type d ! -name all ! -name languages ! -name images | sort` ; do
cd $dir
echo $dir
drush $*
done
</code>
</pre>
h3. d6-all-dump-full.sh
<pre>
<code class="php">
#!/bin/sh
. scripts-config.sh
current_date=`date "+%Y-%m-%d-%Hh%Mm%Ss"`
# Dump D6 database with webassos user.
mysqldump --single-transaction webassos -h myweb.serv.int -u webassos --password=password > $d6_dir_full_backup/webassos/$current_date.webassos.sql
# Dump D6 form database with forum user.
mysqldump forum -h myweb.serv.int -u forum --password=password > $d6_dir_full_backup/forum/$current_date.forum.sql
</code>
</pre>
h3. d6-all-reset-variables.sh
<pre>
<code class="php">
#!/bin/sh
. scripts-config.sh
cd $d6_dir_sites
d6-all-drush.sh vset --always-set reverse_proxy TRUE
d6-all-drush.sh vset --always-set --format=json reverse_proxy_addresses '["147.94.19.16","147.94.19.17"]'
drush @sites -y ev "variable_set('update_notify_emails', array('@email_multi_assos'));"
</code>
</pre>
h3. d6-all-update-contrib.sh
<pre>
<code class="php">
#!/bin/sh
. scripts-config.sh
d6-all-drush.sh -y en update
# Launch cron to check for updates.
d6-all-drush.sh -q cron
d6-all-drush.sh cc all
d6-all-dump-full.sh
d6-all-drush.sh -y upc --no-core
d6-all-drush.sh -y updb
d6-all-drush.sh -y dis update
d6-all-drush.sh cron
</code>
</pre>
h3. d6-dump-individual.sh
<pre>
<code class="php">
#!/bin/sh
. scripts-config.sh
# ARGS: auto or manual, site_prefix
# WARNING : backup are regularly flushed, put your backup in a safe place.
cd $dir_individual_backup
#!/bin/sh
. scripts-config.sh
# ARGS: auto or manual, site_prefix
# WARNING : backup are regularly flushed, put your backup in a safe place.
cd $dir_individual_backup
#récupération des tables du site dans le fichier liste_tables.temp
tables='_%'
liste="$1$tables"
mysql -h myweb.serv.int -u webassos --password=HBVH2ljgyZCA0AP251DY -BNe "show tables like '"$liste"'" webassos | tr '\r\n' ' ' > liste_tables.temp
#transformation de cette liste en une variable
var=$(cat liste_tables.temp)
#sauvegarde de toutes ces tables
current_date=`date "+%Y-%m-%d-%Hh%Mm%Ss"`
suffixe="_dump$current_date.sql"
fichier="$1$suffixe"
if [ $1 = 'auto' ] ; then
mysqldump webassos -h myweb.serv.int -u webassos --password=HBVH2ljgyZCA0AP251DY $var > $d6_dir_individual_auto_backup/$fichier
else
mysqldump webassos -h myweb.serv.int -u webassos --password=HBVH2ljgyZCA0AP251DY $var > $d6_dir_individual_manual_backup/$fichier
fi
#suppression du fichier temporaire utilisé
rm liste_tables.temp
</code>
</pre>
h2. Scripts Drupal 7
h3. d7-all-drush-logged.sh
<pre>
<code class="php">
#!/bin/sh
. scripts-config.sh
. scripts-utils.sh
help="# ARGS: drush_command"
check_arguments $# 1 "$help"
cd $d7_dir_sites
for dir in `find . -maxdepth 1 -mindepth 1 -type d ! -name all | sort` ; do
cd $dir
current_date=`date "+%Y-%m-%d_%Hh:%Mm"`
echo "======$current_date======" >> $d7_dir_log/$dir.log
echo "Arguments are: $* " >> $d7_dir_log/$dir.log
drush $* >> $d7_dir_log/$dir.log
#To have the same caracters limit we needed 'end' that have 4 caracters, we used the Croate language for this ! LOL
echo -e "=============KRAJ=============\n" >> $d7_dir_log/$dir.log
cd -
done
</code>
</pre>
h3. d7-all-drush.sh
<pre>
<code class="php">
#!/bin/sh
. scripts-config.sh
. scripts-utils.sh
help="# ARGS: drush_command"
check_arguments $# 1 "$help"
for dir in `find $d7_dir_sites -maxdepth 1 -mindepth 1 -type d ! -name all | sort` ; do
cd $dir
echo $dir
drush $*
done
</code>
</pre>
h3. d7-all-dump-full.sh
<pre>
<code class="php">
#!/bin/sh
. scripts-config.sh
current_date=`date "+%Y-%m-%d-%Hh%Mm%Ss"`
# Dump D7 databases at once.
mysqldump -h $db_server -u $db_user -ppassword --all-databases > $d7_dir_full_backup/myassos/$current_date.d7_full.sql
</code>
</pre>
h3. d7-all-dump-individual.sh
<pre>
<code class="php">
#!/bin/sh
. scripts-config.sh
. scripts-utils.sh
help="# ARGS: auto or manual"
check_arguments $# 1 "$help"
current_date=`date "+%Y-%m-%d-%Hh%Mm%Ss"`
cd $d7_dir_sites
for dir in `find . -maxdepth 1 -mindepth 1 -type d ! -name all | cut -c3-` ; do
cd $dir
drush cc all
if [ $1 = 'auto' ] ; then
drush sql-dump --result-file="$d7_dir_individual_auto_backup/$dir/$current_date.$dir.sql"
else
drush sql-dump --result-file="$d7_dir_individual_manual_backup/$dir/$current_date.$dir.sql"
fi
cd -
done
</code>
</pre>
h3. d7-all-releases-info.sh
<pre>
<code class="php">
#!/bin/sh
. scripts-config.sh
# List version of a module or theme that exists in sites/all/*
# and all versions that are available for same project on drupal.org
# modules
cd $d7_dir_sites/all/modules
for dir in `ls -1` ; do
if [ -d $dir ] ; then
drush pm-releases $dir
fi
done
# themes
cd $d7_dir_sites/all/themes
for dir in `ls -1` ; do
if [ -d $dir ] ; then
drush pm-releases $dir
fi
done
</code>
</pre>
h3. d7-all-reset-variables.sh
<pre>
<code class="php">
#!/bin/sh
. scripts-config.sh
for dir in `find $d7_dir_sites -maxdepth 1 -mindepth 1 -type d ! -name all | sort` ; do
cd $dir
echo $dir
d7-reset-variables.sh
done
</code>
</pre>
h3. d7-all-security-review.sh
<pre>
<code class="php">
#!/bin/sh
. scripts-config.sh
# grep -v success to keep site_name and error lines.
bash -c "d7-all-drush.sh security-review --store |& grep -v success | mail -s d7-all-security-review.sh $email_multi_assos"
</code>
</pre>
h3. d7-all-update-contrib.sh
<pre>
<code class="php">
#!/bin/sh
. scripts-config.sh
d7-all-drush.sh -y en update
# Launch cron to check for updates.
d7-all-drush.sh -q cron
d7-all-dump-individual.sh auto
d7-all-drush-logged.sh -y upc --no-core
d7-all-drush.sh -y updb
d7-all-drush.sh -y dis update
d7-all-drush.sh cron
</code>
</pre>
h3. d7-all-update-localisation.sh
<pre>
<code class="php">
#!/bin/sh
. scripts-config.sh
d7-all-drush.sh -y en l10n_update
d7-all-drush.sh l10n-update-refresh
d7-all-drush.sh l10n-update
d7-all-drush.sh -y dis l10n_update
</code>
</pre>
h3. d7-create-site.sh
<pre>
<code class="php">
#!/bin/sh
. scripts-config.sh
. scripts-config-site.sh $1
. scripts-utils.sh
help="# ARGS: site name"
######## Exceptions
check_arguments $# 1 "$help"
# "-" is forbidden because it provokes database error.
if [ `echo $1 | grep -` ] ; then
echo '"-" is forbidden in the site name'
exit 1
fi
# Site name length must be lower or equal to 16 due to database limitations.
if [`echo $1 | wc -c` -gt 16 ] ; then
echo "site name can't have more than 16 characters"
exit 1
fi
###### Initialisation
cd $d7_dir
db_password=`ask_password_db $db_server $db_user`
site_password=`generate_password`
# Check if site database already exists.
if mysql -h $db_server -u $db_user -e "USE $d7_site_name" -p$db_password 2>/dev/null ; then
echo "Database $d7_site_name already exists"
exit 1
fi
# Check if site folder already exists.
if [ -d $d7_site_dir ] ; then
echo "Folder $d7_site_dir already exists"
exit 1
fi
###### Main
mkdir $d7_site_dir
dir_site_name="assos.centrale-marseille.fr.$d7_site_name"
# Backup requirements
mkdir $d7_dir_individual_auto_backup/$dir_site_name
mkdir $d7_dir_individual_manual_backup/$dir_site_name
current_date=`date "+%Y-%m-%d-%Hh%Mm%Ss"`
# NB : ls sort by considering the 1st characters
touch $d7_dir_individual_auto_backup/$dir_site_name/$current_date.$dir_site_name.sql
touch $d7_dir_individual_auto_backup/$dir_site_name/$current_date.$dir_site_name.sql2
touch $d7_dir_individual_auto_backup/$dir_site_name/$current_date.$dir_site_name.sql3
# Create and grant privileges on database
mysql -h $db_server -u $db_user -e "CREATE DATABASE $d7_site_name" -p$db_password
mysql -h $db_server -u $db_user -e "GRANT ALL PRIVILEGES ON $d7_site_name.* TO '$d7_site_name'@'%' IDENTIFIED BY '$site_password'" -p$db_password
# Create settings.php
sed "s/\%\%DBUSER\%\%/$d7_site_name/ ; s/\%\%DBNAME\%\%/$d7_site_name/ ; s/\%\%DBPASS\%\%/$site_password/ ; s/\%\%nomsite\%\%/$d7_site_name/" < $d7_settings_template > $d7_site_settings
# Create symbolic link
cd $d7_dir
ln -s . $d7_site_name
# Next Instructions
echo "Go to http://assos.centrale-marseille.fr/$d7_site_name/install.php to continue."
echo "Press enter when ready to go on."
read key
# Init variables
d7-reset-variables.sh $d7_site_name
# Permissions
chmod -R 755 $d7_site_dir
chmod 400 $d7_site_settings
# Last instructions
echo "Last instructions:"
echo "- Advice the webmaster to close account creation on the website"
echo "- Give the webmaster a link to the club Drupal's tutorials "
echo "- Create a node of type \"Site\" on default"
echo "- Register the webmaster on webmasters@listes.centrale-marseille.fr"
</code>
</pre>
h3. d7-delete-site.sh
<pre>
<code class="php">
#!/bin/sh
. scripts-config.sh
. scripts-config-site.sh $1
. scripts-utils.sh
help="# ARGS: site name."
check_arguments $# 1 "$help"
db_password=`ask_password_db $db_server $db_user`
# Delete database.
mysql -h $db_server -u $db_user -p$db_password -e "DROP DATABASE $d7_site_name"
# Delete MYSQL user.
mysql -h $db_server -u $db_user -p$db_password -e "DROP USER '$d7_site_name'@'%'"
# Delete site's folder.
chmod -R 700 $d7_site_dir
rm -r $d7_site_dir
# Delete symbolic link.
rm $d7_dir/$d7_site_name
# Delete database backups.
rm -r $d7_dir_individual_auto_backup/assos.centrale-marseille.fr.$d7_site_name
rm -r $d7_dir_individual_manual_backup/assos.centrale-marseille.fr.$d7_site_name
echo "Don't forget to:"
echo "- Refresh node site on default"
echo "- Check particular behavior (normally detailed on site node)"
</code>
</pre>
h3. d7-reset-variables.sh
<pre>
<code class="php">
#!/bin/sh
. scripts-config.sh
# ARGS: [site_name]
if [ ! -z $1 ] ; then
. scripts-config-site.sh $1
cd $d7_site_dir
fi
current_timestamp=`date "+%s"`
random_1=`dd if=/dev/urandom count=1 2> /dev/null | cksum | cut -f1 -d" "`
random_2=`dd if=/dev/urandom count=1 2> /dev/null | cksum | cut -f1 -d" "`
####### General variables
drush -y vset --always-set reverse_proxy TRUE
drush -y vset --always-set --format=json reverse_proxy_addresses '["147.94.19.16","147.94.19.17"]'
drush -y ev "variable_set('update_notify_emails', array('@email_multi_assos'));"
drush -y vset error_level 0
drush -y vset dblog_row_limit 1000
###### Hidden captcha
drush -y en hidden_captcha
# Log wrong answers.
drush -y vset captcha_log_wrong_responses 1
# Use hidden captcha for all forms.
drush -y sqlq --db-prefix "UPDATE {captcha_points} SET module = 'hidden_captcha', captcha_type = 'Hidden CAPTCHA' WHERE module is NULL;"
# Flush captcha cache.
drush -y vdel captcha_placement_map_cache
# Randomely generate a math question as the label of the hidden captcha field.
drush -y vset hidden_captcha_label "$random_1 + $random_2"
####### Piwik
drush -y en piwik
drush -y vset piwik_site_id "101"
drush -y vset piwik_url_http "http://piwik.centrale-marseille.fr/"
drush -y vset piwik_url_https "https://piwik.centrale-marseille.fr/"
# Piwik cache.
drush -y vset piwik_cache 1
drush -y vset piwik_visibility_roles "1"
# Piwik is enable for everyone except the administrator
drush -y vset --format=json piwik_roles '[3]'
drush -y vset piwik_page_title_hierarchy 1
# Activate local search.
drush -y vset piwik_site_search 1
###### Security review
# For untrusted roles:
# 1: anonymous user
# 2: authenticated user
# 3: administrator
drush -y en security_review
drush -y ev "variable_set('security_review_untrusted_roles', array('1'));"
# Used to initialise entries in the database schema.
drush security-review --store
# Security review can't check for files permissions on multi_assos if launched within the web interface.
drush -y sqlq --db-prefix "UPDATE {security_review} SET skip = '1', skiptime = $current_timestamp, skipuid = '1' WHERE reviewcheck IN ('file_perms');"
drush -y sqlq --db-prefix "UPDATE {security_review} SET skip = '0', skiptime = '0', skipuid = NULL WHERE reviewcheck NOT IN ('file_perms');"
</code>
</pre>
{{>toc}}
h1. Les tâches planifiées
h2. Crontab
Pour exécuter ces tâches régulièrement, on utilise le "crontab":http://fr.wikipedia.org/wiki/Crontab. Il s'agit d'un programme installé sur notre serveur.
Pour voir et modifier la liste des actions :
# se connecter au serveur : <code>ssh assos@sas1.centrale-marseille</code>
# taper la commande pour voir le crontab <code>crontab -l</code>
# taper la commande pour modifier le crontab <code>crontab -e</code>.
{{important(Ne pas effectuer cette opération avant de s'être renseigné sur "vi":http://fr.wikipedia.org/wiki/Vi (l'éditeur de texte utilisé qui n'est pas vraiment intuitif :p) et sur "la syntaxe du crontab":http://fr.wikipedia.org/wiki/Crontab#Modification !
Les scripts doivent si possible s’exécuter avant le club Drupal pour que les problèmes éventuels y soient réglés)}}
{{tip(
* Pour utiliser un autre éditeur que vi taper : export EDITOR=editor_name exemple : vim, nano.
* d7-all-update-contrib.sh fait des sauvegardes automatiques individuelles pour les sites drupal)}}
<pre>
<code class="php">
######## How to use
# mm hh jj MMM JJJ task
# minutes hour day_month month day_week
# for more informations : https://fr.wikipedia.org/wiki/Crontab#Notation
######## End how to
### Cleaning
00 5 * * * /users/guest/assos/bin/all-reset-permissions.sh > /dev/null 2>&1
00 6 * * * /users/guest/assos/bin/delete-temporary-files.sh > /dev/null 2>&1
### Traqu-heure
00 7 * * * /users/guest/assos/bin/traqu_heure/script-restitution.sh > /dev/null 2>&1
0 * * * * /users/guest/assos/bin/traqu_heure/script-traque.sh > /dev/null 2>&1
### Databases
# Create backup
00 8 1,15 * * /users/guest/assos/bin/d7-all-dump-full.sh > /dev/null 2>&1
00 1 * * 2 /users/guest/assos/bin/d6-all-dump-full.sh > /dev/null 2>&1
# flush backup
00 8 2,16 * * /users/guest/assos/bin/flush-full-databases-backups.sh d7 > /dev/null 2>&1
00 22 * * 2 /users/guest/assos/bin/flush-full-databases-backups.sh d6 > /dev/null 2>&1
00 23 * * 2 /users/guest/assos/bin/flush-individual-databases-backups.sh auto d7 > /dev/null 2>&1
00 1 * * 3 /users/guest/assos/bin/flush-individual-databases-backups.sh manual d7 > /dev/null 2>&1
### Drupal 7
00 22 * * 3 /users/guest/assos/bin/d7-all-update-contrib.sh | mail -s "d7-all-update-contrib.sh" assos
00 1 * * 4 /users/guest/assos/bin/d7-all-update-localisation.sh | mail -s "d7-all-update-localisation.sh" assos
00 2 * * 4 /users/guest/assos/bin/d7-all-security-review.sh
00 3 * * 4 /users/guest/assos/bin/d7-all-reset-variables.sh > /dev/null 2>&1
00 4 * * 4 /users/guest/assos/bin/d7-all-releases-info.sh | mail -s "d7-all-releases-info.sh" assos
### Drupal 6
00 22 * * 6 /users/guest/assos/bin/d6-all-update-contrib.sh > /dev/null 2>&1
00 23 * * 6 /users/guest/assos/bin/d6-all-reset-variables.sh > /dev/null 2>&1
# Launch drush cron on fellowings
45 17 * * * /users/guest/assos/bin/drushcronone assos.centrale-marseille.fr.ginfo > /dev/null 2>&1
*/15 * * * * /users/guest/assos/bin/drushcronone assos.centrale-marseille.fr.clubfinance > /dev/null 2>&1
45 3 * * * /users/guest/assos/bin/drushcronone default > /dev/null 2>&1
35 00 * * * /users/guest/assos/bin/drushcronone assos.centrale-marseille.fr.cac13 > /dev/null 2>&1
*/45 * * * * /users/guest/assos/bin/drushcronone assos.centrale-marseille.fr.expat > /dev/null 2>&1
*/10 * * * * /users/guest/assos/bin/drushcronone forum.centrale-marseille.fr > /dev/null 2>&1
00 22 * * 7 /users/guest/assos/bin/disk-usage.sh | mail -s "disk-usage.sh" assos
</code>
</pre>
La liste des tâches effectuées régulièrement est ci-dessous.
h2. La réinitialisation des variables
Cette tâche consiste à réinitialiser certaines variables qui donnent des droits considérés comme trop permissifs donc dangereux aux administrateurs des sites ou nécessaire pour d'autres scripts tels que piwik ou security_review.
En voici quelques exemples :
<pre>
<code class="bash">
drush -y vset error_level 0
</code>
</pre>
Cette commande permet de ne pas afficher les messages d'erreurs. En effet, ils contiennent parfois des informations sensibles sur l'installation et ne doivent donc pas être divulgués à n'importe qui.
<pre>
<code class="php">
drush -y vset --always-set reverse_proxy TRUE
drush -y vset --always-set --format=json reverse_proxy_addresses '["147.94.19.16","147.94.19.17"]'
</code>
</pre>
Ces commandes permettent de déclarer à Drupal les serveurs proxy du CRI afin d'éviter qu'il ne répertorie tous les visiteurs comme ayant l'adresse des serveurs sus-cités. **Pas fini : voir http://assos.centrale-marseille.fr/lessive/content/t%C3%A2che/rendre-les-settingsphp-conformes-%C3%A0-la-version-712-de-drupal**
h2. La sauvegarde des bases de données
Sur les installations d6 et d7 : une fois par semaine
On utilise les [[Scripts_et_taches_planifiees#d6-all-dump-fullsh|d6-all-dump-full.sh]], [[Scripts_et_taches_planifiees#d6-dump-individualsh|d6-dump-individual.sh]], [[Scripts_et_taches_planifiees#d7-all-dump-fullsh|d7-all-dump-full.sh]] et [[Scripts_et_taches_planifiees#d7-all-dump-individualsh|d7-all-dump-individual.sh]].
h2. La purge des sauvegardes de bases de données bdd
Sur les installations d6 et d7 : toutes les semaines
Cette tâche utilise les scripts le script de purges [[Scripts_et_taches_planifiees#purge_des_sauvegardes.sh|purge des sauvegardes ([[Scripts_et_taches_planifiees#flush-full-databases-backupssh|flush-full-databases-backups.sh]] et [[Scripts_et_taches_planifiees#flush-individual-databases-backupssh|flush-individual-databases-backups.sh]]) sauvegardes]] afin de libérer de l'espace disque en supprimant les sauvegardes de bdd les plus vieilles.
h1. Liste des scripts à disposition
Les scripts utilisés sont hébergés dans le répertoire bin du compte assos.
h2. Configuration et utilitaires
h3. scripts-config.sh
<pre>
<code class="php">
#!/bin/sh
PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/games:/usr/local/sbin:/usr/local/bin:/users/guest/assos/bin
PATH=$PATH:/usr/guest/assos/bin
scripts_config='imported'
email_multi_assos="assos@centrale-marseille.fr"
dir_multi_assos='/users/guest/assos'
dir_scripts="$dir_multi_assos/bin"
dir_template="$dir_multi_assos/template"
###### Database
db_server="myassos.serv.int"
db_user="assos"
dir_log="$dir_multi_assos/log"
###### Backup
db_full_backup_number=3
db_individual_manual_backup_number=2
db_individual_auto_backup_number=3
dir_backup="$dir_multi_assos/backup"
dir_full_backup="$dir_backup/full"
dir_individual_backup="$dir_backup/individual"
dir_individual_manual_backup="$dir_individual_backup/manual"
dir_individual_auto_backup="$dir_individual_backup/auto"
###### D7 variables
d7_dir="$dir_multi_assos/htmltest"
d7_dir_sites="$d7_dir/sites"
### Log
d7_dir_log="$dir_log/d7"
### Backup
d7_dir_full_backup="$dir_full_backup/d7"
d7_dir_individual_manual_backup="$dir_individual_manual_backup/d7"
d7_dir_individual_auto_backup="$dir_individual_auto_backup/d7"
### Template
d7_settings_template_name="settings-D7-bddinde-template.php"
d7_settings_template="$dir_template/$d7_settings_template_name"
###### D6 variables
d6_dir="$dir_multi_assos/html"
d6_dir_sites="$d6_dir/sites"
### Backup
d6_dir_full_backup="$dir_full_backup/d6"
d6_dir_individual_manual_backup="$dir_individual_manual_backup/d6"
d6_dir_individual_auto_backup="$dir_individual_auto_backup/d6"
</code>
</pre>
h3. scripts-config-site.sh
<pre>
<code class="php">
#!/bin/sh
# Sets useful variables of a site.
# ARGS: site_name
# Requires scripts-config.sh
# Check if scripts-config.sh is imported.
if [ -z $scripts_config ] ; then
echo "Import of scripts-config.sh required."
exit 1
fi
if [ -z $1 ] ; then
echo "This script needs a site name as argument."
exit 1
fi
scripts_config_site='imported'
d7_site_name=$1
d7_site_dir="$d7_dir_sites/assos.centrale-marseille.fr.$d7_site_name"
d7_site_settings="$d7_site_dir/settings.php"
</code>
</pre>
h3. scripts-utils.sh
<pre>
<code class="php">
#!/bin/sh
# This script contains useful functions for other scripts.
# Check if scripts-config.sh is imported.
if [ -z $scripts_config ] ; then
echo "Import of scripts-config.sh required."
. scripts-config.sh
fi
scripts_utils='imported'
ask_password() {
# read -s doesn't work with sh.
# usage: pass=`ask_password "password please:"`
echo $1 >&2
echo -n ">" >&2
stty_avant=`stty -g`
stty -echo
read password
stty $stty_avant
echo "$password"
unset password
}
ask_password_db() {
# ARGS: server_name, user_name
local db_password="pour_boucler"
# empty db request to validate password
while ! mysql -h $1 -u $2 -p$db_password -e "" 2>/dev/null ; do
db_password=`ask_password "database password:"`
done
echo $db_password
}
generate_password(){
# ARGS: [password_length]
# The password contains special characters. '/' must be excluded to avoid sed malfunction.
local site_password='/'
if [ -z $1 ] ; then
local password_length=20
else
local password_length=$1
fi
while echo "$site_password" | grep -Fq '/' ; do
site_password=`dd if=/dev/urandom count=1 | uuencode -m - | head -n 2 | tail -n 1 | cut -c-$password_length`
done
echo $site_password
}
count_d7_sites(){
find $d7_dir_sites -type d ! -name all -maxdepth 1 | wc -l
}
check_arguments() {
# ARGS: number of arguments passed to script, number of arguments required, [help text]
if [ $1 -lt $2 ] ; then
echo "Number of arguments insuffisant."
echo $3
exit 1
fi
}
</code>
</pre>
h2. Scripts généraux
h3. all-reset-permissions.sh
<pre>
<code class="php">
#!/bin/sh
. scripts-config.sh
# This script puts the correct permissions to sites folders, settings.php and scripts.
######### drupal 6
for dir in `find $d6_dir_sites -type d -maxdepth 1 -mindepth 1 ! -name all ! -name images ! -name languages` ; do
chmod 755 $dir
chmod 400 $dir/settings.php
done
######### drupal 7
for dir in `find $d7_dir_sites -type d -maxdepth 1 -mindepth 1 ! -name all` ; do
chmod 755 $dir
chmod 400 $dir/settings.php
done
####### bin
chmod -R 700 $dir_scripts
####### backup
chmod -R 700 $dir_backup
####### log
chmod -R 700 $dir_log
</code>
</pre>
h3. contrib-usage.sh
<pre>
<code class="php">
#!/bin/sh
. scripts-config.sh
. scripts-utils.sh
help="# ARGS: Drupal_version, project_status, project_name\n
# List sites that have the project_name with the corresponding project_status.\n
# project status: enabled or disabled"
check_arguments $# 3 "$help"
if [ $1 = d6 ] ; then
cd $d6_dir_sites
else
cd $d7_dir_sites
fi
for dir in `find . -maxdepth 1 -mindepth 1 -type d ! -name all ! -name languages ! -name images | sort ` ; do
# List non-core projects that correspond to the status.
# Keep project_name if listed.
# Count line result. 0 if not listed or 1 if listed.
# Print site_dir if listed.
cd $dir;
if [ 1 -eq `drush pml --no-core --status=$2 | grep $3 | wc -l` ] ; then
echo $dir;
fi
cd -
done
</code>
</pre>
h3. delete-temporary-files.sh
<pre>
<code class="php">
#!/bin/sh
. scripts-config.sh
find $dir_multi_assos -name '*~' -delete
</code>
</pre>
h3. disk-usage.sh
<pre>
<code class="php">
#!/bin/sh
. scripts-config.sh
echo -e "Quotas: 9.1 GB\n"
echo "Assos:"
du -hcs $dir_multi_assos | grep total
echo -e "\n"
echo "D6:"
cd $d6_dir_sites
du -h -d 1
echo -e "\n"
echo "D7:"
cd $d7_dir_sites
du -h -d 1
echo -e "\n"
echo "Logs:"
du -hcs $dir_log | grep total
echo -e "\n"
echo "Backups:"
cd $dir_backup
du -h -d 2
</code>
</pre>
h3. drushcronone
<pre>
<code class="php">
#!/bin/sh
. scripts-config.sh
if [ $# -lt 1 ]; then
echo "usage: $0 <site's folder name>"
exit 1
fi
cd "$d6_sites_dir/$*"
pwd;
drush cron;
</code>
</pre>
h3. flush-files.sh
<pre>
<code class="php">
#!/bin/sh
help="# ARGS: number of files to keep, [email address]\n
# usage: flush files from a folder except a number equal to argument."
# Check if scripts-config.sh is imported.
if [ -z $scripts_utils ] ; then
echo "Import of scripts-utils.sh required."
. scripts-utils.sh
fi
check_arguments $# 1 "$help"
backups_number=`ls | wc -l`
let "number_of_backups_to_delete = backups_number - $1" > /dev/null
if [ $number_of_backups_to_delete -gt 0 ] ; then
ls | head -$number_of_backups_to_delete | xargs rm
else
if [ ! -z $2 ] ; then
dir=`pwd`
echo "There are not enough files in $dir to Flush it. Check if backup script works fine." | mail -s "[db] $dir has a backup problem" $2
fi
fi
</code>
</pre>
h3. flush-full-databases-backups.sh
<pre>
<code class="php">
#!/bin/sh
. scripts-config.sh
# ARGS: Drupal version
cd $dir_full_backup/$1
for dir in `ls` ; do
cd $dir
flush-files.sh $db_full_backup_number $email_multi_assos
cd -
done
</code>
</pre>
h3. flush-individual-databases-backups.sh
<pre>
<code class="php">
#!/bin/sh
. scripts-config.sh
# ARGS: manual or auto, Drupal version
cd $dir_individual_backup/$1/$2
for dir in `ls` ; do
cd $dir
if [ $1 = 'auto' ] ; then
flush-files.sh $db_individual_auto_backup_number $email_multi_assos
else
flush-files.sh $db_individual_manual_backup_number
fi
cd -
done
</code>
</pre>
h2. Scripts Drupal 6
h3. d6-all-drush.sh
<pre>
<code class="php">
#!/bin/sh
. scripts-config.sh
. scripts-utils.sh
help="# ARGS: drush_command"
check_arguments $# 1 "$help"
for dir in `find $d6_dir_sites -maxdepth 1 -mindepth 1 -type d ! -name all ! -name languages ! -name images | sort` ; do
cd $dir
echo $dir
drush $*
done
</code>
</pre>
h3. d6-all-dump-full.sh
<pre>
<code class="php">
#!/bin/sh
. scripts-config.sh
current_date=`date "+%Y-%m-%d-%Hh%Mm%Ss"`
# Dump D6 database with webassos user.
mysqldump --single-transaction webassos -h myweb.serv.int -u webassos --password=password > $d6_dir_full_backup/webassos/$current_date.webassos.sql
# Dump D6 form database with forum user.
mysqldump forum -h myweb.serv.int -u forum --password=password > $d6_dir_full_backup/forum/$current_date.forum.sql
</code>
</pre>
h3. d6-all-reset-variables.sh
<pre>
<code class="php">
#!/bin/sh
. scripts-config.sh
cd $d6_dir_sites
d6-all-drush.sh vset --always-set reverse_proxy TRUE
d6-all-drush.sh vset --always-set --format=json reverse_proxy_addresses '["147.94.19.16","147.94.19.17"]'
drush @sites -y ev "variable_set('update_notify_emails', array('@email_multi_assos'));"
</code>
</pre>
h3. d6-all-update-contrib.sh
<pre>
<code class="php">
#!/bin/sh
. scripts-config.sh
d6-all-drush.sh -y en update
# Launch cron to check for updates.
d6-all-drush.sh -q cron
d6-all-drush.sh cc all
d6-all-dump-full.sh
d6-all-drush.sh -y upc --no-core
d6-all-drush.sh -y updb
d6-all-drush.sh -y dis update
d6-all-drush.sh cron
</code>
</pre>
h3. d6-dump-individual.sh
<pre>
<code class="php">
#!/bin/sh
. scripts-config.sh
# ARGS: auto or manual, site_prefix
# WARNING : backup are regularly flushed, put your backup in a safe place.
cd $dir_individual_backup
#!/bin/sh
. scripts-config.sh
# ARGS: auto or manual, site_prefix
# WARNING : backup are regularly flushed, put your backup in a safe place.
cd $dir_individual_backup
#récupération des tables du site dans le fichier liste_tables.temp
tables='_%'
liste="$1$tables"
mysql -h myweb.serv.int -u webassos --password=HBVH2ljgyZCA0AP251DY -BNe "show tables like '"$liste"'" webassos | tr '\r\n' ' ' > liste_tables.temp
#transformation de cette liste en une variable
var=$(cat liste_tables.temp)
#sauvegarde de toutes ces tables
current_date=`date "+%Y-%m-%d-%Hh%Mm%Ss"`
suffixe="_dump$current_date.sql"
fichier="$1$suffixe"
if [ $1 = 'auto' ] ; then
mysqldump webassos -h myweb.serv.int -u webassos --password=HBVH2ljgyZCA0AP251DY $var > $d6_dir_individual_auto_backup/$fichier
else
mysqldump webassos -h myweb.serv.int -u webassos --password=HBVH2ljgyZCA0AP251DY $var > $d6_dir_individual_manual_backup/$fichier
fi
#suppression du fichier temporaire utilisé
rm liste_tables.temp
</code>
</pre>
h2. Scripts Drupal 7
h3. d7-all-drush-logged.sh
<pre>
<code class="php">
#!/bin/sh
. scripts-config.sh
. scripts-utils.sh
help="# ARGS: drush_command"
check_arguments $# 1 "$help"
cd $d7_dir_sites
for dir in `find . -maxdepth 1 -mindepth 1 -type d ! -name all | sort` ; do
cd $dir
current_date=`date "+%Y-%m-%d_%Hh:%Mm"`
echo "======$current_date======" >> $d7_dir_log/$dir.log
echo "Arguments are: $* " >> $d7_dir_log/$dir.log
drush $* >> $d7_dir_log/$dir.log
#To have the same caracters limit we needed 'end' that have 4 caracters, we used the Croate language for this ! LOL
echo -e "=============KRAJ=============\n" >> $d7_dir_log/$dir.log
cd -
done
</code>
</pre>
h3. d7-all-drush.sh
<pre>
<code class="php">
#!/bin/sh
. scripts-config.sh
. scripts-utils.sh
help="# ARGS: drush_command"
check_arguments $# 1 "$help"
for dir in `find $d7_dir_sites -maxdepth 1 -mindepth 1 -type d ! -name all | sort` ; do
cd $dir
echo $dir
drush $*
done
</code>
</pre>
h3. d7-all-dump-full.sh
<pre>
<code class="php">
#!/bin/sh
. scripts-config.sh
current_date=`date "+%Y-%m-%d-%Hh%Mm%Ss"`
# Dump D7 databases at once.
mysqldump -h $db_server -u $db_user -ppassword --all-databases > $d7_dir_full_backup/myassos/$current_date.d7_full.sql
</code>
</pre>
h3. d7-all-dump-individual.sh
<pre>
<code class="php">
#!/bin/sh
. scripts-config.sh
. scripts-utils.sh
help="# ARGS: auto or manual"
check_arguments $# 1 "$help"
current_date=`date "+%Y-%m-%d-%Hh%Mm%Ss"`
cd $d7_dir_sites
for dir in `find . -maxdepth 1 -mindepth 1 -type d ! -name all | cut -c3-` ; do
cd $dir
drush cc all
if [ $1 = 'auto' ] ; then
drush sql-dump --result-file="$d7_dir_individual_auto_backup/$dir/$current_date.$dir.sql"
else
drush sql-dump --result-file="$d7_dir_individual_manual_backup/$dir/$current_date.$dir.sql"
fi
cd -
done
</code>
</pre>
h3. d7-all-releases-info.sh
<pre>
<code class="php">
#!/bin/sh
. scripts-config.sh
# List version of a module or theme that exists in sites/all/*
# and all versions that are available for same project on drupal.org
# modules
cd $d7_dir_sites/all/modules
for dir in `ls -1` ; do
if [ -d $dir ] ; then
drush pm-releases $dir
fi
done
# themes
cd $d7_dir_sites/all/themes
for dir in `ls -1` ; do
if [ -d $dir ] ; then
drush pm-releases $dir
fi
done
</code>
</pre>
h3. d7-all-reset-variables.sh
<pre>
<code class="php">
#!/bin/sh
. scripts-config.sh
for dir in `find $d7_dir_sites -maxdepth 1 -mindepth 1 -type d ! -name all | sort` ; do
cd $dir
echo $dir
d7-reset-variables.sh
done
</code>
</pre>
h3. d7-all-security-review.sh
<pre>
<code class="php">
#!/bin/sh
. scripts-config.sh
# grep -v success to keep site_name and error lines.
bash -c "d7-all-drush.sh security-review --store |& grep -v success | mail -s d7-all-security-review.sh $email_multi_assos"
</code>
</pre>
h3. d7-all-update-contrib.sh
<pre>
<code class="php">
#!/bin/sh
. scripts-config.sh
d7-all-drush.sh -y en update
# Launch cron to check for updates.
d7-all-drush.sh -q cron
d7-all-dump-individual.sh auto
d7-all-drush-logged.sh -y upc --no-core
d7-all-drush.sh -y updb
d7-all-drush.sh -y dis update
d7-all-drush.sh cron
</code>
</pre>
h3. d7-all-update-localisation.sh
<pre>
<code class="php">
#!/bin/sh
. scripts-config.sh
d7-all-drush.sh -y en l10n_update
d7-all-drush.sh l10n-update-refresh
d7-all-drush.sh l10n-update
d7-all-drush.sh -y dis l10n_update
</code>
</pre>
h3. d7-create-site.sh
<pre>
<code class="php">
#!/bin/sh
. scripts-config.sh
. scripts-config-site.sh $1
. scripts-utils.sh
help="# ARGS: site name"
######## Exceptions
check_arguments $# 1 "$help"
# "-" is forbidden because it provokes database error.
if [ `echo $1 | grep -` ] ; then
echo '"-" is forbidden in the site name'
exit 1
fi
# Site name length must be lower or equal to 16 due to database limitations.
if [`echo $1 | wc -c` -gt 16 ] ; then
echo "site name can't have more than 16 characters"
exit 1
fi
###### Initialisation
cd $d7_dir
db_password=`ask_password_db $db_server $db_user`
site_password=`generate_password`
# Check if site database already exists.
if mysql -h $db_server -u $db_user -e "USE $d7_site_name" -p$db_password 2>/dev/null ; then
echo "Database $d7_site_name already exists"
exit 1
fi
# Check if site folder already exists.
if [ -d $d7_site_dir ] ; then
echo "Folder $d7_site_dir already exists"
exit 1
fi
###### Main
mkdir $d7_site_dir
dir_site_name="assos.centrale-marseille.fr.$d7_site_name"
# Backup requirements
mkdir $d7_dir_individual_auto_backup/$dir_site_name
mkdir $d7_dir_individual_manual_backup/$dir_site_name
current_date=`date "+%Y-%m-%d-%Hh%Mm%Ss"`
# NB : ls sort by considering the 1st characters
touch $d7_dir_individual_auto_backup/$dir_site_name/$current_date.$dir_site_name.sql
touch $d7_dir_individual_auto_backup/$dir_site_name/$current_date.$dir_site_name.sql2
touch $d7_dir_individual_auto_backup/$dir_site_name/$current_date.$dir_site_name.sql3
# Create and grant privileges on database
mysql -h $db_server -u $db_user -e "CREATE DATABASE $d7_site_name" -p$db_password
mysql -h $db_server -u $db_user -e "GRANT ALL PRIVILEGES ON $d7_site_name.* TO '$d7_site_name'@'%' IDENTIFIED BY '$site_password'" -p$db_password
# Create settings.php
sed "s/\%\%DBUSER\%\%/$d7_site_name/ ; s/\%\%DBNAME\%\%/$d7_site_name/ ; s/\%\%DBPASS\%\%/$site_password/ ; s/\%\%nomsite\%\%/$d7_site_name/" < $d7_settings_template > $d7_site_settings
# Create symbolic link
cd $d7_dir
ln -s . $d7_site_name
# Next Instructions
echo "Go to http://assos.centrale-marseille.fr/$d7_site_name/install.php to continue."
echo "Press enter when ready to go on."
read key
# Init variables
d7-reset-variables.sh $d7_site_name
# Permissions
chmod -R 755 $d7_site_dir
chmod 400 $d7_site_settings
# Last instructions
echo "Last instructions:"
echo "- Advice the webmaster to close account creation on the website"
echo "- Give the webmaster a link to the club Drupal's tutorials "
echo "- Create a node of type \"Site\" on default"
echo "- Register the webmaster on webmasters@listes.centrale-marseille.fr"
</code>
</pre>
h3. d7-delete-site.sh
<pre>
<code class="php">
#!/bin/sh
. scripts-config.sh
. scripts-config-site.sh $1
. scripts-utils.sh
help="# ARGS: site name."
check_arguments $# 1 "$help"
db_password=`ask_password_db $db_server $db_user`
# Delete database.
mysql -h $db_server -u $db_user -p$db_password -e "DROP DATABASE $d7_site_name"
# Delete MYSQL user.
mysql -h $db_server -u $db_user -p$db_password -e "DROP USER '$d7_site_name'@'%'"
# Delete site's folder.
chmod -R 700 $d7_site_dir
rm -r $d7_site_dir
# Delete symbolic link.
rm $d7_dir/$d7_site_name
# Delete database backups.
rm -r $d7_dir_individual_auto_backup/assos.centrale-marseille.fr.$d7_site_name
rm -r $d7_dir_individual_manual_backup/assos.centrale-marseille.fr.$d7_site_name
echo "Don't forget to:"
echo "- Refresh node site on default"
echo "- Check particular behavior (normally detailed on site node)"
</code>
</pre>
h3. d7-reset-variables.sh
<pre>
<code class="php">
#!/bin/sh
. scripts-config.sh
# ARGS: [site_name]
if [ ! -z $1 ] ; then
. scripts-config-site.sh $1
cd $d7_site_dir
fi
current_timestamp=`date "+%s"`
random_1=`dd if=/dev/urandom count=1 2> /dev/null | cksum | cut -f1 -d" "`
random_2=`dd if=/dev/urandom count=1 2> /dev/null | cksum | cut -f1 -d" "`
####### General variables
drush -y vset --always-set reverse_proxy TRUE
drush -y vset --always-set --format=json reverse_proxy_addresses '["147.94.19.16","147.94.19.17"]'
drush -y ev "variable_set('update_notify_emails', array('@email_multi_assos'));"
drush -y vset error_level 0
drush -y vset dblog_row_limit 1000
###### Hidden captcha
drush -y en hidden_captcha
# Log wrong answers.
drush -y vset captcha_log_wrong_responses 1
# Use hidden captcha for all forms.
drush -y sqlq --db-prefix "UPDATE {captcha_points} SET module = 'hidden_captcha', captcha_type = 'Hidden CAPTCHA' WHERE module is NULL;"
# Flush captcha cache.
drush -y vdel captcha_placement_map_cache
# Randomely generate a math question as the label of the hidden captcha field.
drush -y vset hidden_captcha_label "$random_1 + $random_2"
####### Piwik
drush -y en piwik
drush -y vset piwik_site_id "101"
drush -y vset piwik_url_http "http://piwik.centrale-marseille.fr/"
drush -y vset piwik_url_https "https://piwik.centrale-marseille.fr/"
# Piwik cache.
drush -y vset piwik_cache 1
drush -y vset piwik_visibility_roles "1"
# Piwik is enable for everyone except the administrator
drush -y vset --format=json piwik_roles '[3]'
drush -y vset piwik_page_title_hierarchy 1
# Activate local search.
drush -y vset piwik_site_search 1
###### Security review
# For untrusted roles:
# 1: anonymous user
# 2: authenticated user
# 3: administrator
drush -y en security_review
drush -y ev "variable_set('security_review_untrusted_roles', array('1'));"
# Used to initialise entries in the database schema.
drush security-review --store
# Security review can't check for files permissions on multi_assos if launched within the web interface.
drush -y sqlq --db-prefix "UPDATE {security_review} SET skip = '1', skiptime = $current_timestamp, skipuid = '1' WHERE reviewcheck IN ('file_perms');"
drush -y sqlq --db-prefix "UPDATE {security_review} SET skip = '0', skiptime = '0', skipuid = NULL WHERE reviewcheck NOT IN ('file_perms');"
</code>
</pre>