# Them, All Magazine Migration Changelog
Welcome to the Them, All Magazine migration Changelog, a chronicle of our audacious journey. Here, we not only document code transformations but also share spirited dialogues, hectic problem-solving, and how we collectively configured our project: migrating Them, All Magazine's hosting to Adele, a feminist server maintained by Systerserver.
Our path unfolds in five acts of defiance:
*Briefing Current Redundant Citadel*
Delve into the stronghold of the existing website, initially crafted with Node.js—a powerful but overkill tool for our purpose. It was originally built using a Glitch template, featuring a complex structure of just-js-it-all. After thinking together, we decided to simplify the website to pure HTML and CSS, meticulously tie up each folder with images, audio, and other resources.
*Preparing Our Feminist Forge*
Prepare the environment within Systerserver: Configure the web server, install essential requirements, and lay the foundation for the migration.
*Crafting the Cipher of Release*
Create and maintain an SSL certificate to implement the HTTPS protocol, ensuring smooth browsing for Them, All Magazine without encountering warnings.
*GitLab Onboarding*
Navigate code updates and documentation in Systerserver's GitLab.
*Test test testing*
Conduct testing on GitLab uploads for online presence via the Hook!
## [0.0.0] - 2023-06-21
GATHERED: Meeting and Project Overview
We gathered online through FuturÉtic's Jitsi instance. During the meeting, R provided an overview of the project, outlined the necessary steps, and shared essential information, details, and materials to start the project. We also examined the existing website hosting and delved into the code structure using Node.js.
## [0.0.1] - 2023-07-04
READ:
- GitLab Server Hooks Documentation
- GitLab Gitaly Troubleshooting Documentation
- GitLab Omnibus Maintenance Documentation
CHECKED:
*What is a Hook? What Hook Do We Need?*
A Git hook is a script triggered by specific Git events, like committing or pushing, allowing custom actions before or after these events. In our case, a hook is needed that activates after code is uploaded or edited in the Git repository; this specific hook is called the 'post-receive' hook.
*Is Gitaly Running in the GitLab Instance?*
It's essential to verify if Gitaly is running since it's a critical component responsible for Git data storage and access. Hooks in GitLab may depend on Gitaly to function.
*What Are the Prerequisites to Create a Hook?*
Prerequisites include the storage name, path to the Gitaly configuration file, and the repository's relative path.
*Where in the Server Is GitLab Installed?*
GitLab is located at `/opt/gitlab`.
*Where Is the Gitaly Configuration File?*
The Gitaly configuration file is located at `/var/opt/gitlab/gitaly/config.toml`.
TRIED OUT:
*Restart Unicorn*
We executed the command `sudo gitlab-ctl restart unicorn` to restart Unicorn, allowing us to run Gitaly again. Gitaly relies on the Unicorn service, which is responsible for running the Unicorn web server serving GitLab.
*Checking If Gitaly Is Up*
To check the status of Gitaly, we ran the command `sudo gitlab-ctl status`. The result confirmed that Gitaly service is running with the process ID (PID) 4259, and the log service associated with Gitaly has the PID 4250.
*Locate the Hooks Directory in the GitLab Server*
- We navigated to the relevant directory using the command `cd /opt/gitlab/embedded/service/`. Inside this directory, there is "gitlab-shell," which typically contains the "hooks" directory.
- While trying to access the "gitlab-shell" directory, we encountered an error related to Ruby versions. To address this, we ran the command `rvm install "ruby-3.0.5"`.
- After successfully installing Ruby, we could navigate to "gitlab-shell," but it was observed that there was no "hooks" directory inside. Running the command `ls` confirmed the absence of the "hooks" directory.
*Since There Is No Hooks Directory, We Created a "custom_hooks" Repo Inside the Path of the Project's .gits*
The command used for this action was `mkdir custom_hooks`.
NOTED:
The command `systemctl status gitlab-gitaly.service` is not applicable to newer versions of GitLab because it uses a different service management system. In newer versions, GitLab typically uses the `gitlab-ctl` command.
## [0.0.2] - 2023-07-05
CHATTED: Server Hook Progress
A: Heyo, how was yesterday? I got closer with the server hook but still not enough...
E: I didn't go any further. How was Gitaly?
A: Gitaly was just running. We had seen it before, and I also forgot. We just couldn't see it with the service command because that's for older GitLab versions.
CHATTED: Late-Night Work
E: I had to stay up all night.
A: Same, I planned today to work on the hook and couldn't. Not the best day. If you feel tired now, maybe it's better to sleep and pick it up tomorrow fresh.
E: I had a meeting in the morning and had to finish the other project. SWEL!
A: Nooooo
EMAILED: Webhook Update
A: Hello R, Unfortunately, we didn't manage to finish yet with the webhook, and we would like to postpone our meeting to next week. Is that possible for you?
R: Hi A and E, No problem! Next Thursday, 13th July at 1 pm is good too! Warmest, Rxx
## [0.0.3] - 2023-07-06
EMAILED: Code and Content Update
R: Hi A and E, I made a few changes to the pages, so it might be good to work from a new tar zip? Attached also are the background images I added to the projects, contact, and podcast pages. I hope that's okay. I'll send you the new code over WeTransfer too. Best, Rx
CHATTED: Project Progress
A: Heylo, cycled a lot today, saw much sand.
E: Sounds chill. Did we say to meet for a work session? I don't see it on my terminal.
A: Hmm, only an online meeting with R. Let's continue asynchronously and report.
## [0.0.4] - 2023-07-10
CHATTED: Feeling Stuck
E: Are you up for a meetup tomorrow?
A: We have a Debian upgrade at Varia today; it might take long hours.
E: But maybe the upgrade goes smoothly; we will have alcohol?
A: Yeah, could be, but I don't have high hopes, hahaha.
E: Ooo.
A: My brain is a bit slow.
E: Hhh, we meet about Borguser soon; I need to eat.
A: Eat before that... The test is working, the file is visible, blablabla. It should work. The script manually works, but it doesn't get automatically triggered, and I don't see results online. Pff, it makes me so tired.
E: Okay, so the same situation as with my webhook tests, stuck at the trigger.
A: What could be wroooong? Yikes.
E: Kjkavjsdlkmd.
READ:
- GitLab Server Hook Creation Guide
CHECKED:
*Setting Up a Server-Side Hook for a Repository*
To set up a server-side hook for a repository, follow these steps:
1. In our GitLab's instance interface, we go to Admin area > Projects and select the project to which we want to add a server hook.
2. We locate the Gitaly relative path on the page that appears. This is where the server hook must be implemented.
3. On the file system, we create a new directory in this location called custom_hooks.
4. Inside the new custom_hooks directory, we create a file with a name matching the hook type. In this case, the filename should be post-receive with no extension.
5. We make the hook file executable and ensure that it’s owned by the Git user.
6. We write the code to make the server hook function as expected. Hooks can be in any language. We ensure the shebang at the top properly reflects the language type. For example, if the script is in Bash, the shebang is `#!/bin/bash`.
TRIED OUT:
*Inside the new custom_hooks directory, create the hook file without an extension.*
Run the command: `touch custom_hooks/post-receive`.
*In the Gitaly config file, check the storage name and the relative path for the git repository. In our server, the config.toml is located at: /var/opt/gitlab/gitaly/.*
Run the command: `cat config.toml` to read the content of the file. There is:
[[storage]]
name = "default"
path = "/var/opt/gitlab/git-data/repositories"
*Create a repository for them-all magazine in /var/www/ for the web server. this will be served online.*
NOTED:
We initially doubted where to place the script: in the web server's folder /var/www/them-all-magazine or the Gitaly relative path /var/opt/gitlab/git-data/repositories/@hashed/. Finally, we realized the script is linked to the Git instance on the server. It triggers after changes in the Git repo, directing how the web server executes a Git pull, making the website live. Clearing things up required careful reading and research to understand the essential steps for the hook to function and the website to be served from our web server.
CHECKED:
*Small Guide for Next Steps*
Adele, maintained by Systerserver, hosts our GitLab and Apache web server, with the following process flow:
1. Create a dedicated folder on our GitLab server for Them-All-Magazine website files, including HTML, CSS, fonts, and images. R, the magazine editor, can then upload and edit files through GitLab's interface.
2. Create a server hook script on our GitLab server by placing it in the custom_hooks folder within our Git repository. Access Adele to determine the correct relative path and create the hook file.
3. Create a repository on Adele at /var/www/ to host Them-All-Magazine website content. This standard directory ensures easy access, and initializing it as a Git repository allows remote content pulling from our primary GitLab repository.
4. In the hook script, add a `git pull` command from /var/www/projects/ to automatically update website content.
5. Configure the Web Server: Finally, adjust the configuration of our Apache web server on Adele to ensure that the Them-All-Magazine website is accessible online.
*How to Configure Web Server?*
Configure Apache web server on Adele to handle incoming web requests by creating `them-all-http.conf` and `them-all-https.conf` configuration files, specifying rules and settings.
*Write configuration files*
`them-all-http.conf`
This configuration file defines a virtual host for HTTP traffic (port 80).
<VirtualHost *:80>
ServerName them-all-magazine.com
ServerAlias www.them-all-magazine.com
DocumentRoot /var/www/them-all-magazine
<Directory /var/www/them-all-magazine>
Options +ExecCGI
AddHandler cgi-script .cgi .sh
Options Indexes FollowSymLinks ExecCGI
AllowOverride All
Require all granted
</Directory>
</VirtualHost>
`them-all-https.conf`
This configuration file defines a virtual host for HTTPS traffic (port 443).
<IfModule mod_ssl.c>
<VirtualHost *:443>
ServerName them-all-magazine.com
ServerAlias www.them-all-magazine.com
DocumentRoot /var/www/them-all-magazine
SSLEngine on
SSLCertificateFile /etc/letsencrypt/live/them-all-magazine.com/fullchain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/them-all-magazine.com/privkey.pem
<Directory /var/www/them-all-magazine>
Options +ExecCGI
AddHandler cgi-script .cgi .sh
Options Indexes FollowSymLinks ExecCGI
AllowOverride All
Require all granted
</Directory>
</VirtualHost>
</IfModule>
*Handling DNS and Let's Encrypt*
To make the website accessible via the domain name (them-all-magazine.com), we need to configure DNS settings. We have to create DNS records that point our domain to the IP address of our server. This can be done through the domain registrar's control panel.
For Let's Encrypt, the steps include:
- Install Certbot, the Let's Encrypt client, on our server.
- Use Certbot to request an SSL certificate for the domain we want. We used the command: `certbot certonly --webroot -w /var/www/them-all-magazine -d them-all-magazine.com -d www.them-all-magazine.com`
## [0.0.5] - 2023-07-11
EMAILED:
Many back-and-forth emails to try to find a date to meet, according to different schedules, precarious freelance work, PhD pressures, life.
CHATTED:
A: There is a possibility that it is not our fault but a GitLab version issue. Anyway, I found one new approach that I will try...
E: I also saw information about a webhook version
## [0.0.6] - 2023-07-12
CHATTED:
A: Hey, all good?
E: Did you manage to chill?
A: Yep, although I had a headache and toothache, haha.
E: Is it gone already?
A: Yeah, mostly. It's raining. I'm going to exercise. Afterwards, I'll try one more thing for the hook. Worst-case scenario, we put the code anyway, and when it changes, we git pull. I'm saying it's a quick fix if we fail. What are your plans for today?
E: That IS the worst! We take turns looking at the Git repo, exercise, cat sit, finish zine. How about you?
A: I've done exercising and some chores. Now I'm halfway through the hook.
E: I'm also done with the physical activity, and now I'm going for cat sitting.
## [0.0.7] - 2023-07-13
CHATTED:
E: I'm going to manually upgrade the webhook today. Some people say only an upgrade helped. From 2.6.9 to 2.8.1 (latest).
A: Ok, let's hope it will help. If we don't solve it soon, we can at least have the code in GitLab, show R how to edit files, and git pull on the server. Till we fix...
E: Yes, sounds like a plan.
Webhook Issue. Gonna try now quickly.
A: I have to leave. I copied HTML/CSS in Git but they need cleaning. They are weirdly formatted because of the Node.js.
E: I upgraded the webhook but nothing. Why doesn't it show up on our dev server?
A: I don't know! Probably some misconfiguration in Apache. Bah!
E: systemctl reload apache2.service - failed because a fatal signal was delivered to the control process. Restart works smoothly. Reload was a headache.
A: Hahah... What is life?
E: I wanna cry.
A: Nah, it's okay. Nobody will die from this, no worries, we'll fix it. Everything is ok.
E: Pfff: misconfiguration in Apache - wrong path in HTTP & HTTPS conf.
A: Yeah... Sorry, probably I did that during my failed attempts.
E: Really doubt that; it looks like from our first attempts. Gonna delete the wrong/extra folder on the server too. Alright.
A: Yep, purge, throw it into the ocean. Did you git pull everything?
E: Editing and pulling. The syntax is so annoying. Ok: restructured the system > added more elaborate meta tags > nav was inside the everywhere, so moved it inside the body > downloaded images, audio, and linked them in HTMLs.
A: Cool!
E: It could be that because this repo is private, webhooks don't work? Meh, I made my Gitea repo private, and now it's just asking for my password.
A: We could make it public and see. Now, going home after babysitting.
E: ...and, of course, it's not that simple with GitLab, surprise. Whyyy?
A: Complicated, documentation like treasure hunting, pfff.
E: No, wait, I did it. Gonna test now.
A: Sending luck.
E: Nope, no, nah.
A: Bah, for a moment, I believed it.
E: I didn't]]]]
## [0.0.8] - 2023-07-14
READ:
- GitLab Server Hooks Documentation
- Automatically Deploy Page on GitLab with Git Hooks
TRIED OUT:
*Write / edit post-receive hook.*
*Check hook from Gitlab's side first.*
*Try to trigger the hook, see what happens from the web server's side.*
*Debug. Find the errors that appear when trying to trigger the hook.*
*Version 1 (Basic Script):*
`post-receive`
This script is a simple post-receive hook that changes the working directory to the deployment location and then updates the local repository by pulling the latest changes from the adequate remote Git repository.
#!/bin/bash
# Change to the desired deployment directory
cd /var/www/them-all-magazine
# Perform a git pull to update the repository
sudo git pull
It would be wonderful if this code just worked, but after testing, it did not :) Realization: while functional, this script lacks error handling and feedback on whether the `git pull` was successful. So we decided to create a second version to provide better visibility into the script's execution and capture error messages for troubleshooting and maintenance.
*Version 2 (Enhanced Script with Logging):*
`post-receive`
This version adds logging to the `/tmp/hook.log` file. This helps in tracking the execution of the script, which is useful for debugging. By redirecting both standard output and standard error to the log file, we ensure that any errors during the `git pull` are recorded. The use of a log file makes it easier to review problems or verify that the script executed as expected.
#!/bin/bash
# Create a log file for tracking script execution
touch /tmp/hook.log
# Log the start of the script execution
echo "Executing post-receive hook..." >> /tmp/hook.log
# Change to the desired deployment directory
cd /var/www/them-all-magazine
# Log the beginning of the Git pull operation
echo "Pulling latest changes..." >> /tmp/hook.log
# Perform a git pull and log the output, including errors
sudo /usr/bin/git pull >> /tmp/hook.log 2>&1
# Log the completion of the script
echo "Post-receive hook execution completed." >> /tmp/hook.log
*How to test the hook?*
- Trigger the Hook: To test the hook, simply push code changes to the GitLab repository, which should trigger the post-receive hook. This can be done by making changes to a branch and then pushing those changes to GitLab.
- Check the Hook's Output: The post-receive hook script should log its output to the hook.log file or another specified location. This log file will contain information about the execution of the hook, including any errors or success messages.
- Review the Hook's Effect: Check the effect of the hook. In our case check if the website is updating. Verify that this happens as a result of the hook execution.
*Errors Logged in the hook.log File:*
fatal: this operation must be run in a work tree
>> This error shows that the git pull operation must be performed within a Git working tree.
There is no tracking information for the current branch.
Please specify which branch you want to merge with.
>> This error to specify the git branch explicitly
git pull
If you wish to set tracking information for this branch you can do so with:
git branch --set-upstream-to=/ main
fatal: not a git repository: '/var/www/them-all-magazine'
>> This error occurs because the script attempted a Git operation in a directory that is not
>> a Git repository, resulting in the "not a git repository" message.
Host key verification failed.
fatal: Could not read from remote repository.
>> These errors are related to authentication issues. They indicate that Git couldn't verify
>> the host key and couldn't read from the remote repository
>> often because of access rights or authentication problems.
Please make sure you have the correct access rights
and the repository exists.
*Steps that Resolved the Issues:*
- Change Ownership of the Folder to be Git: Changed the ownership of the `/var/www/them-all-magazine` folder to the Git user. This is necessary to allow Git to perform operations in that directory.
- Add Git User to Sudoers: Since the `/var/www` directory was owned by root, Git didn't have the necessary permissions to execute `git pull`. To grant Git the required privileges, added the Git user to the sudoers file.
- Edit the Sudoers File: Added the Git user to the sudoers file in the `/etc/sudoers.d/` directory using the command `nano gitlab-postreceive-hook` and included the content: `git ALL=(ALL:ALL) NOPASSWD: /usr/bin/git pull`. This line permits the Git user to execute the specific command `git pull` with elevated privileges without requiring a password.
- Successful Hook Execution: After making these changes, the post-receive hook was executed successfully without encountering any errors.
!!!
After following these steps, the Git user had the necessary permissions and privileges to perform a git pull operation in the specified directory, resolving the issues logged in the hook.log file and enabling a successful hook execution
!!!
## [0.0.9] - 2023-07-17
EMAILED:
- ALERTA ALERTA HOOK WORKS WITHOUT BUGGY BUGYS!
- Happy back-and-forth emails about our mellow work together <3
- Reminded of Caladona meetup, where we met R for the first time
## [0.1.0] - 2023-07-18
EMAILED:
- Edited and committed changes in GitLab, they appear online thanks to the hook :)
- Arranged invoices because we need to pay rents and go to the market for food
~The end