not_gitmodules

Not Gitmodules

Introduction

Not Gitmodules is a lightweight, production-friendly and open-source Python utility designed to simplify managing external modules in your project.
It’s a replacement for Git Submodule which is: up to 10x faster, safer, simpler, cleaner and the most importantly - without headaches.

Still have questions?

👉 Watch the introduction video on YouTube, where Snoop Dogg will gently explain everything!

Effortlessness

Everything you need to do is:

Install Not Gitmodules. Clone this repo from GitHub or install from PyPi
Create a simple YAML file.

Not Gitmodules will handle the rest for you.

Why Not Gitmodules?

Simplicity: Minimalistic design and no unnecessary complexities.
Production-Ready: Explicitly licensed for production use.
Dependency-Free: Uses only Python’s built-in tools.
OS-Agnostic: Works seamlessly on Linux, MacOS and any other platforms where Python is available by default.

Usage

Part 1: Installation

Choose the most convenient way to install Not Gitmodules:

Clone (or download) this repository and include it to your project and use as a part of your project’s code:
```
git clone git@github.com:Armen-Jean-Andreasian/not_gitmodules.git
```
Install via a package manager and use as a Python package (example with pip):
```
 pip install not-gitmodules
```
- And, yeah, don’t forget to add it to requirements.txt in advance, if you don’t use Poetry.

Part 2: Preparation

Create a YAML file in your project’s root directory.

Tip: Naming it notgitmodules.yaml lets you use it without explicitly specifying it in the command.

Define the submodules following the pattern:

folder_to_save: # Destination folder for this group of modules
  module1: module1.url # directory_name: url (ssh or https)
  module2: module2.url

another_folder:
  module3: module3.url

Example:

# directory_name: url (ssh or https)  
utils:
  file_manager: https://github.com/not-gitmodules/notgitmodules-file-manager-py
  file_encryptor: https://github.com/not-gitmodules/notgitmodules-file-encryptor-py

services:
  forsaken_mail: https://github.com/malaohu/forsaken-mail
  sim_mail: https://github.com/Webador/SlmMail

The example above will download and create this structure:

utils/
    file_manager/
        ... (containing of the repo)
    file_encryptor/
        ...

services:
    forsaken_mail/
        ...
    sim_mail/
        ...

Part 3: Installation

Flag (all of them are optional)	Description	Example
`-y`, `--yaml-path`	Specify a custom path for the YAML file. By default, it looks for `notgitmodules.yaml` in the current working directory. Naming it `notgitmodules` is a matter of best practices; you can name it as you want.	• `not_gitmodules -y /path/to/custom_notgitmodules.yaml`: Uses a custom YAML file located at `/path/to/custom_notgitmodules.yaml`
`-t`, `--threaded` (default behaviour)	Enable threaded execution, where repositories are cloned in parallel (using threads). This flag is mutually exclusive with `-s`. This is the default behavior if neither `-t` nor `-s` is specified.	• `not_gitmodules -t`: Clones repositories in parallel using threads • `not_gitmodules --threaded`: Same as `-t`, using long form
`-s`, `--sequential`	Enable sequential execution, where repositories are cloned one by one in the order they appear in the YAML file. This flag is mutually exclusive with `-t`.	• `not_gitmodules -s`: Clones repositories one by one in order • `not_gitmodules --sequential`: Same as `-s`, using long form

More command examples:

Default command:

This will look for notgitmodules.yaml in the project root and create a directory named my_gitmodules in the root to download the modules into, in parallel mode using threads.

not_gitmodules install

Command pattern:

not_gitmodules install --yaml-path </path/to/notgitmodules.yaml>  --<execution_type> 

not_gitmodules install -y </path/to/notgitmodules.yaml>  --sequential

Note: Usually with undefined amount of workers in ThereadPool in parallel mode take more than 52% less time than in parallel mode.

Part 4. Dockerizing

FROM python:3.10-slim  

# Install git for not_gitmodules
RUN apt-get update && apt-get install -y git  

RUN pip install --no-cache-dir -r requirements.txt   

# copy the notgitmodules.yaml file (default). Modify accordingly.
COPY notgitmodules.yaml .

# install modules using not_gitmodules.
RUN not_gitmodules install -y notgitmodules.yaml -t

WORKDIR /app  
  
COPY . .

CMD ["python", "main.py"]

Good to Know

Not Gitmodules doesn’t require to keep the folders with modules. You can safely .gitignore or delete them.
Do not use matching names for the repositories in notgitmodules.yaml file. In that case only the first repository will be downloaded and the second one will be skipped.
Not Gitmodules needs Git and PyYAML for functioning.
Not Gitmodules, just like Gitmodules, doesn’t automatically install the dependencies of submodules (such as requirements.txt for Python or package.json for JavaScript).
Not Gitmodules doesn’t download the sub-dependency submodules (if they are not defined properly for Git).
- Reason: it’s practically inefficient, may lead to bugs and bottlenecks, and better to maintain manually.
- Solution: Include the sub-dependency submodule to YAML file manually.
Not Gitmodules keeps the view of keeping the project structure clean. All submodules among one project/microservice need to go to one folder. It’s recommended to use dependency injection in case of having nested not_gitmodules.
Possible bottleneck with private repositories.
- If cloning fails, but you have access to the repository, provide the HTTPS repo URL instead of SSH url in YAML file.
If your python version is incompatible with PyYAML==6.0 try to install the compatible version of PyYAML.

This project is licensed under a Custom License. See the LICENSE file for full details.

Key points:

You may use this project for commercial or personal purposes.

Author

Armen-Jean Andreasian, 2024

This site is open source. Improve this page.