Posted by: Odzangba | October 13, 2006

How to build local APT repositories

I was going through my /var/cache/apt/archives folder the other day and I thought to myself, “So will I have to re-download all these packages if I do a clean install? It’d be cool to build my own APT repository with the 3000+ packages in my local cache.” I mean, internet access is not cheap here in Ghana and the speed is nothing to write home about. The only snag was I didn’t know how to do it. So I went hunting on google and it turned out a lot has been published on this topic. I found it all rather confusing, mostly geeks talking to other geeks in geekish, so I decided to write my own how-to for my much simpler mind.

If your roaring to go and know what you’re doing, skip down to the summary. If you want to take things slow however, read on. Okay, enough talking already, let’s get our hands dirty…

1) Create folders for the package files you plan to keep in your repository. Here is an example: in your home folder, create a new folder called repository in which you will keep all your downloaded packages. If you plan to burn the package files onto a CD, I suggest you create separate folders for each disk. The naming is up to you – disk_1, disk_2, etc (I’ll assume this naming convention throughout the rest of the tutorial). This is just to ensure that you use the next folder once your cache reaches the size of a CD (usually 700MB). If you are using a local hard drive or a DVD, you obviously have higher limits to keep in mind. If you do not plan to create a CD/DVD, you may simply dump all the files in the repository folder.

2) The next step is to copy all your deb files (those files that end with .deb)  to the repository folder(s). Open Nautilus, navigate to /var/cache/apt/archives and copy all the deb files to the appropriate folder(s). For instance, /home/<username>/repository/disk_1 – keep an eye on the size of the folders. To do it in the terminal:

cp /var/cache/apt/archives/*.deb ~/repository/disk_1 

If you have many files, it may take quite a while so be patient.

3) Change into your repository folder:

cd ~/repository/disk_1

4) Now do the following command to create the Packages.gz file that is needed to for Synaptic to “see” your repository:

sudo dpkg-scanpackages . /dev/null | gzip -9c > Packages.gz

I suggest you copy and paste this to avoid any typos.

  • Please note that every time you add any more .deb files to this folder, you have to create a new Packages.gz file using the above command before the new file(s) will show up in Synaptic (or Aptitude).
  • Be sure to install the build-essential package (sudo aptitude install build-essential) before running the above command.

There are several ways of using your newly created repository.

I) Keeping the files on a local hard disk…

Edit your /etc/apt/sources.list file like so:

gksudo gedit /etc/apt/sources.list

And insert this on a new line (preferably the first):

deb file:/home/username/repository/disk_1/ /

Remember to replace username with your real username.

Reload your package index like this:

sudo aptitude update

II) Using a CD/DVD as a repository:

a) Burn the repository folder onto a CD/DVD

b) With the disc loaded in your drive, fire up Synaptic and click through the menus like this:

Edit –> Add CD Rom.

c) You will be asked to type in a description for the disc; type in anything, for instance: Offline Repository Disk 1.

d) Click Ok.

Summary
Go to /var/cache/apt/archives and copy your debian packages to a folder of your choice, for example, /home/<username>/repository/

  • Change into the repository directory

cd /home/username/repository

  • And generate a Packages.gz file like this:

sudo dpkg-scanpackages . /dev/null | gzip -9c > Packages.gz

Make sure build-essential is installed (sudo aptitude install build-essential) before you run the above command.

  • Add the following line to your sources.list file (/etc/apt/sources.list)

deb file:/home/username/repository/ /

Remember to replace username with your real username

  • Reload your package index like this:

sudo apt-get update

That’s it… sit back, relax and enjoy the ride.

I found the following tips for using Synaptic very useful:

  • You might want to check out the online sources.list generator. It will let you select various repositories for your sources.list. Once you get the output, simply replace the contents of your /etc/apt/sources.list file with it.
  • Now have a loom at the Synaptic settings (Settings –> Repositories –> Settings).
  • I suggest you decide if you always want to get the highest/latest version or if you want to stick to the feisty version.
  • Also, ensure that you have checked the option that allows downloaded packages to be kept in the cache, otherwise you will just have to download them again if you accidentally do an apt-get autoclean or something similar. Have fun. :)

Cheers,
Odzangba


Responses

  1. [...] By localized I don’t mean ‘on a server in the same country’ although that would be a start. Imean something more along the lines of a CD/DVD/External hard drive set up as a package repository. Currently I create custom repositories myself following this guide by another Linux Accra member. I’d love for there to be a faster way to do this though. Maybe I should look into creating a tool to automate the custom repository creation. Or does one exist already? I just wish there was some  awareness on the part of the people who are trying to promote open source in developing countries that bandwidth can be a huge issue here and can affect how ‘free’ something is vs. readily available pirated software. [...]

  2. Even here in rural USA bandwidth is slow, thanks for the info.

  3. Fred,
    I’m glad my how-to was of any help :)

  4. Hey your info on How to build local APT repositories has made me had bad headache. Nothing works right on Ubuntu Dapper.

    You might need to make some correction aboutpackages directories:

    ## My local repository
    deb file: /home/“username”/repository/disk_1/

    The right thing is;
    deb file: ///home/“username”/repository disk_1/

    Huh… agree???

  5. Event the
    ## My local repository
    deb file: /home/“username”/repository disk_1/

    After googling here and there, eventually I found the working repository setting.

    http://www.ubuntuforums.org/archive/index.php/t-7455.html

    I have proved it, and it just woks so well.

    1) Create a dir in your home (ex. /home/pinco/Ubuntu-repository)

    2) Generate Package file: dpkg-scanpackages . /dev/null > Packages

    3) Insert in sources.list the correctly path like this:

    deb file:/home/pinco/Ubuntu-repository ./

    4) sudo apt-get update

  6. goutzilla,
    The line in my how-to is correct:

    ## My local repository
    deb file: /home/“username”/repository/disk_1 /

    You forgot to insert a spcae between the “disk_1″ and the final “/”. That’s why I encouraged you to copy and paste to avoid typos. :) I’m glad you finally got it to work though.

    Cheers

  7. hey dude thnx a lot its really good support for me but i think there is something missing, i.e. we also have to copy repository files to /var/cache/apt/archives
    P.S. ive written a script for u guys help enjoy ;)

    #dpkg-scanpackages . /dev/null | gzip -9c > Packages.gz

    cp sources.list /etc/apt/
    cp -pr repository/ /root/
    cp repository/* /var/cache/apt/archives/
    aptitude update
    echo ‘***********successfull***********’

  8. [...] repository server is done. If you want more information, I found this page very helpful. Now make sure apache is running and we can move on to the final step, making your [...]

  9. This is so cool. I always thought you could use apt with local deb files. I never thought it would be this easy. Far better and quicker than redownloading EVERYTHING! This is especially useful for me. I gathered my /var/cache/apt/archives from my first ubuntu install. And then I took my archives from the cache that was backed up in my other linux installs (kubuntu, knoppix). I stored it all on my external usb harddrive and now I can install everything that I love. Far better than the tie it takes to download games and things like that. LONG LIVE OPEN SOURCE SOFTWARE :-) thx for this tutorial!

  10. This is very useful. It saves time and in your case money. I’ll just like to add a little thing I had to do before I was able to do this. I am on Ubuntu 7.10 Gutsy, however the same applies to Ubuntu 7.04 Feisty and maybe other versions. Ubuntu doesn’t have the command: ‘dpkg-scanpackages’ installed. So just run this:

    $ sudo apt-get install dpkg-dev

    Just type in your password and stuff, and let it install. That’ll get you dpkg-scanpackages and more dpkg utilities. I hope I helped. Also I know on the Gutsy LiveCD has the dpkg-dev and apt-get will ask you to insert the install cd. Other versions of Ubuntu I am not sure.

  11. It’s probably a better idea to install the build-essential package as one of the first things you do after installing ubuntu. Just do

    sudo apt-get install build-essential

    That will save you a lot of headache later on.

    Ciao,
    Odzangba

  12. I have an offline machine without a DVD-ROM but with content of 4 DVDs of packages on its hdd, and I’ve been wondering how to make synaptic see those packages. Hope, your post will help.

  13. Hi Xemul,
    Good luck with the offline repository. Post back here if you run into any problems.

    Cheers,
    Odzangba

  14. its very cool viva forums.

  15. was looking for exactly this info. beautifully explained. thanks!

  16. Have fun with it. :-D

  17. Nice explication for creating his own repository.

    But, one question : there is none gpg authentification for this repository, and then, each time that an install is realised (with synaptic, or apt-get), an error message appear.

    Do you know how to solve this problem ?

  18. hey thanks a lot … thanks a lot now I can backup and share my downloaded packages

  19. Hey. This is a great tutorial. Thanks. Do you know the Nonetdebs project? http://nonetdebs.homeip.net/. This is great if you have a laptop with no internet connection and want to install things on it. I have put my repository on an usb memory stick :-)

  20. nice nice neat
    jut tell me will it work with RPMs and synaptic?

  21. of course if all deb instances are replaced with rpm

  22. i don’t think this will work for rpm.

    createrepo

    should work and is the better way, not to mix things up (go the way your distribution wants you to go)
    (createrepo must be installed ;o]). if successful you should get the metadata “repodata” in .

    besides this command also works as well:

    dpkg-scanpackages ./ /dev/null > Packages

    it doesn’t need to be gzipped as long as it’s local.

    (be aware to do all as root, otherwise it could end in an local exploitable point if the package can be written by the world [for k/ubuntu users sudo all]l)

    greetz.

  23. @umair syed

    you should put
    #!/bin/sh
    as first line. oh and you should just echo “succsessful” if there’s really no error, so test it. keep things clear.
    furthermore wtf shall that cp sources.list /etc/apt, you won’t have a sources.list but a Packages.gz argh, there you go:

    #!/bin/sh
    touch error_of_the_script_xxyyzz
    dpkg-scanpackages ./ /dev/null | gzip -9c > Packages.gz 2> error_of_the_script_xxyyzz
    sizeoferrorfile=$(du error_of_the_script_xxyyzz | cut -b 1)
    if [ $sizeoferrorfile -eq 0 ]
    then
    echo -ne “\ndeb file:$(pwd)” // >> /etc/apt/sources.list
    echo ‘***********successfull***********’
    else
    echo ‘ERROR: ‘
    cat error_of_the_script_xxyyzz
    fi
    rm error_of_the_script_xxyyzz
    #if you want the following command to be also executed just remove the #
    #apt-get update

    you could place it in /usr/sbin
    it has the be run in the desired directory.
    don’t forget to “chmod +x script_name.sh” to make it executable.
    when moving the packages directory you have to edit your /etc/apt/sources.list

    GreetZ.

  24. This is close to what Im tryin gto find. Simply put Im trying to find a totally offline way of using a 120 gb usb hard drive as my software sources for all of my repo disks. The DVD’s are frankly getting worn out. I need a way to store my repo DVD’s in one place so the computer can look in one place while Im updating things, or installing software while mobile and not online.

    Please email me asap before my disks finally wear out! LOL thanks!

  25. Great guide, thanks a lot :) I have been doing several Debian installs on virtual machines recently and this saved a lot of bandwidth! I now also have a ‘base’ repository on a USB key so getting from a fresh install to a finished system is much faster now.
    o/

  26. thanks guys..
    i was looking for this..

  27. Thanks HEAPS for this post. I was stuck trying to do this for hours… still learning. ;-)

    http://ubuntuforums.org/showthread.php?p=5653789#post5653789

  28. You may want to try apt-cacher.

  29. Thanks Dear. I was looking for this info. for a long time. I don’t have an internet connection in my pc. But I can download .deb packages from another computer. So this will be helpfull.

  30. If i wanted to backup my apt Repository to a different directory on my computer, what path would i set in the sources.list file.

  31. Ricardo, I’m not sure I understand but you would use the path to the backup location in your sources.list file.

  32. thanks for the info.
    but i think aptoncd is better then whole of this procedure

  33. odzangba could i take the repo folder i made and just put it in a repo directory online like my website and have a deb http://mysite.com/repo/ would this work for a online repo to add to source list and install software?

  34. were is the package zip file located when it done being created?

  35. it in the /home/your-name/ folder this is were your package.gz file will be once its created.

    LOL i found it haha!

    • Glad you found it :) And yes could put the repo online and it would work.

  36. Hi
    I like your page and the way your explain things.
    I’m a newbie, so some stupid questions.

    I have three computers, all of them are old 800 Mhz.
    Sorry this is the luxury I have. I have a very slow internet connection. I ISP provides me with an IP.
    So basically I have all my Ubuntu’s on 192.168.1.X network and I connected them all to a switch.

    1) Could you explain how I can convert one Ubuntu into a repo for my other two.
    2) Force the other two to use the local repo rather than download from the internet.
    3) What should the sources.list look like on the local as well as the remote.

    Eagerly waiting for your help.

  37. Hi koolneze, it seems during one of my late night clean ups, I inadvertently removed the section on using the mirror over a local network… I don’t know what I was thinking. :D Luckily, I have a similar post here http://odzangba.wordpress.com/2007/12/24/use-apt-mirror-to-create-your-own-ubuntu-mirror/

    Scroll down to the “Method Two” section and follow the instructions. Essentially, you’ll have to install a web server (apache) on one of the machines and tell the others to only download packages from it. Good luck. :)

  38. Here is some info i thought you would like if you want to know how repos work and how to make one on line.

    http://www.jejik.com/articles/2006/09/setting_up_and_managing_an_apt_repository_with_reprepro/

  39. I was wailting for this a long time.thanks for your help.i want to know “do it works if i dont copy the deb files in /var/cache/apt/archive/” ?

  40. [...] [upmod] [downmod] How to build local APT repositories « Odzangba Kafui Dake’s Blog (odzangba.wordpress.com) 0 points posted 1 year, 1 month ago by trshant tags 4mdelicious howto [...]


Leave a response

Your response:

Categories