Skip to content

installation

Jump to bottom
Ryan Hodges edited this page Jan 5, 2022 · 65 revisions

Installation

Audience: Client IT, Developers

Production

Installation on Ubuntu 20.04 LTS (22.04 LTS and 18.04 LTS should be very similar)

  1. Create the directory /usr/local/apps/

    sudo mkdir /usr/local/apps
cd /usr/local/apps
sudo chmod 775 /usr/local/apps
sudo chgrp adm /usr/local/apps
    • NOTE: The above assumes that your user has sudo privileges and is part of the adm group

  2. Install Dependencies

    sudo apt-get update
sudo apt-get install git python3 python3-pip python3-virtualenv virtualenv gcc make uwsgi uwsgi-plugin-python3 gdal-bin python3-gdal python3-dev build-essential -y

  3. Clone repository into /usr/local/apps/

    git clone https://github.com/Ecotrust/TEKDB.git

  4. Create and activate a Python3 Virtual Environment

    cd /usr/local/apps/TEKDB/
python3 -m pip install --user virtualenv
virtualenv env --python=python3
source /usr/local/apps/TEKDB/env/bin/activate

  5. Install Postgres + PostGIS

    sudo apt install postgresql-12 postgresql-contrib postgresql-server-dev-12 postgis postgresql-12-postgis-3 -y

  6. Create Database and DB User

    Replace USERNAME with an appropriate username for your database user below. You will be prompted to provide a new password as well.

    sudo -u postgres createuser -s -P USERNAME
sudo -u postgres createdb -O USERNAME tekdb
sudo -u postgres psql -c "CREATE EXTENSION postgis; CREATE EXTENSION postgis_topology;" tekdb

  7. Install Python dependencies

    pip install -r /usr/local/apps/TEKDB/TEKDB/requirements.txt

  8. Enable External Access to Postgres Enable authentication from remote access

    < FOR Ubuntu 16.04 >

    sudo vim /etc/postgresql/9.6/main/pg_hba.conf

    < FOR Ubuntu 18.04 >

    sudo vim /etc/postgresql/10/main/pg_hba.conf

    < FOR Ubuntu 20.04 >

    sudo vim /etc/postgresql/12/main/pg_hba.conf

    Find the section of uncommented lines near the bottom.

    Update direct access for your local postgres user (change 'peer' to 'trust'):

    local   all             postgres                                trust

    Add the following line to the Authentication section at the bottom:

    host    tekdb           USERNAME                                md5

    Enable remote access to PostgreSQL server:

    < FOR Ubuntu 18.04 >

    sudo vim /etc/postgresql/10/main/postgresql.conf

    < FOR Ubuntu 20.04 >

    sudo vim /etc/postgresql/12/main/postgresql.conf

    Uncomment the 'listen_addresses' line and change it to read as such:

    listen_addresses = '*'

    Restart PostgreSQL Server:

    sudo service postgresql restart

  9. Install Proj.4 You can install from source, as GeoDjango recommends, or just us a package manager (untested):

    sudo apt install libproj-dev proj-bin -y

  10. Update Local Settings

    cp /usr/local/apps/TEKDB/TEKDB/TEKDB/local_settings.py.template /usr/local/apps/TEKDB/TEKDB/TEKDB/local_settings.py
vim /usr/local/apps/TEKDB/TEKDB/TEKDB/local_settings.py
    • You may use whatever text editor you prefer: emacs, nano, vim, etc...
    • Set the following:
      • ALLOWED_HOSTS: add your domain name to the list, i.e.:
      [ 
  'localhost',
  'your.site.com'
]

      - NOTE: a value of '*' will allow any requests to and all urls
      • default_lon: the longitude of the center of your study area in EPSG:3857 coordinates (get a GIS tech to help you with this)
      • default_lat: the latitude of the center of your study area in EPSG:3857 coordinates
      • default_zoom: 'Guess and check' - find the zoom level that shows your whole study area in the app's map view window. Here is an example of what these numbers look like:
        • 0: the whole world
        • 3: The U.S.A. or a small continent
        • 6: A medium-large U.S. state
        • 10: A county
        • 12: A forest
      • SECRET_KEY: make something long and complex - you will not need to remember or enter it, just be sure not to share it.
      • DATABASES: 
        {
   'default': {
      'ENGINE': 'django.contrib.gis.db.backends.postgis',
      'NAME': 'tekdb',
      'USER': 'USERNAME',
      'PASSWORD': 'PASSWORD',
      'HOST': 'localhost',
      'PORT': 5432
   }
}

  11. [OPTIONAL] DJ Shortcut Many of the commands below assume an activated virtual environment and explicitly call the manage.py script. You can save yourself some typing in the future by replaceing these commands with the alias 'dj' for 'manage.py':

    sudo vim /etc/bash.bashrc

    Then add the following to the bottom of the file:

    alias dj="/usr/local/apps/env/bin/python /usr/local/apps/TEKDB/TEKDB/manage.py"
alias djrun="dj runserver 0.0.0.0:8000"

  12. Run migration

    python /usr/local/apps/TEKDB/TEKDB/manage.py migrate

  13. Run static file collection

    python /usr/local/apps/TEKDB/TEKDB/manage.py collectstatic

  14. Create Superuser (or import legacy data)

    • If building a new database without data to load into it
    python /usr/local/apps/TEKDB/TEKDB/manage.py loaddata /usr/local/apps/TEKDB/TEKDB/TEKDB/fixtures/default_users_fixture.json
    • This will create the default groups:
      • Administrator
      • Editor
      • ReadOnly
    • This will also create three default users (with corresponding permissions)
      • admin
      • editor
      • readonly
    • Reach out to your administrator for default passwords and edit these as soon as possible
    • If migrating old MTKEDB data in, [follow steps in this document].(https://github.com/Ecotrust/TEKDB/wiki/Migrating-From-Access-to-Postgres)

  15. Load Initial Lookup Data

    • The database administration forms include numerous dropdown choices. While all of these choices may be populated by hand, it's better to start with them primed with some of the more common choices. To do this, simply run:
    python /usr/local/apps/TEKDB/TEKDB/manage.py loaddata /usr/local/apps/TEKDB/TEKDB/TEKDB/fixtures/default_lookups_fixture.json

  16. Update your Database iterators A problem on older installs is that after importing data into the database, the iterators may not be updated (or maybe are overwritten by the data import). Rather than check that every single iterator sequence in the database matches the IDs that exist in their sister tables, we've written a handy script:

    1. If the database schema has changed since you last generated the sequence updater script, create a new one:

      cd /usr/local/apps/TEKDB/deployment
sudo -u postgres psql -Atq tekdb -f /usr/local/apps/TEKDB/deployment/generate_db_iterator_reset.sql  -o /usr/local/apps/TEKDB/deployment/reset_db_iterators_YYYYMMDD.sql

      Be sure to replace YYYYMMDD with today's date and remove any old versions of the script.

    2. If the 'reset_db_iterators...' script is up to date, run it:

      sudo -u postgres psql tekdb -f /usr/local/apps/TEKDB/deployment/reset_db_iterators_YYYYMMDD.sql

  17. Test your installation with Django’s Dev Server

    • If you need to debug and test here:
      1. Make sure network traffic is allowed on port 8000
      2. run python /usr/local/apps/TEKDB/TEKDB/manage.py runserver 0.0.0.0:8000
      3. Know the IP address (or URL if you have DNS set up) of your server
      4. punch your site's address at port 8000 into a browser, i.e.: your.domain.com:8000

  18. Install and configure NGINX

    sudo apt-get install nginx -y
sudo cp /usr/local/apps/TEKDB/deployment/tekdb_nginx.conf /etc/nginx/sites-available/tekdb
sudo rm /etc/nginx/sites-enabled/default
sudo ln -s /etc/nginx/sites-available/tekdb /etc/nginx/sites-enabled/tekdb
sudo cp /usr/local/apps/TEKDB/deployment/uwsgi_params /etc/nginx/

  19. Configure UWSGI and boot proceses:

    sudo cp /usr/local/apps/TEKDB/deployment/emperor.ini /etc/uwsgi/
sudo ln -s /usr/local/apps/TEKDB/deployment/uwsgi.service /etc/systemd/system/
sudo ln -s /usr/local/apps/TEKDB/deployment/tekdb.ini /etc/uwsgi/apps-enabled/
sudo service uwsgi start
sudo service uwsgi restart
sudo cp /usr/local/apps/TEKDB/deployment/rc.local /etc/rc.local

  20. 18.04 and other Systemd-only OSes:

    sudo cp /usr/local/apps/TEKDB/deployment/rc.local.service /etc/systemd/system/rc.local.service
sudo chmod 744 /etc/rc.local
sudo systemctl enable rc.local

  21. Set Media Folder Permissions:

    sudo groupadd mediausers
sudo adduser www-data mediausers
sudo chgrp -R mediausers /usr/local/apps/TEKDB/TEKDB/media
sudo chmod -R 770 /usr/local/apps/TEKDB/TEKDB/media

  22. Reboot

    sudo reboot 0

  23. Test Server and Networking/Ports

Automatic (Unattended) Security Updates

From the document Using the "unattended-upgrades" package

Install the unattended-upgrades package if it isn't already installed (sudo apt-get install unattended-upgrades).

To enable it, do:

sudo dpkg-reconfigure --priority=low unattended-upgrades

(it's an interactive dialog) which will create /etc/apt/apt.conf.d/20auto-upgrades with the following contents:

APT::Periodic::Update-Package-Lists "1";
APT::Periodic::Unattended-Upgrade "1";

To have the server automatically reboot when necessary to install security upddates:

  1. install the package update-notifier-common 
sudo apt-get install update-notifier-common
  1. edit the file /etc/apt/apt.conf.d/50unattended-upgrades near the bottom you will find the line
//Unattended-Upgrade::Automatic-Reboot "false";

uncomment it and set value to true:

Unattended-Upgrade::Automatic-Reboot "true";

To tell the server what time is most safe to reboot (when needed), uncomment the line

//Unattended-Upgrade::Automatic-Reboot-Time "02:00";

And set the time to your desired restart time.

Read the source document for more details.

Debugging

  • startup scripts logs: /var/log/rc.local/log
  • nginx error log: /var/log/nginx/tekdb.error.log
  • nginx access log: /var/log/nginx/tekdb.access.log

Development:

  1. Create a DEVELOPMENT DIRECTORY

    1. this doc will assume Unix formatted addresses:
      1. if running Windows, you will use \ instead of /
    2. Put this somewhere where your user has full privileges, like one of the following examples:
      1. Linux: /home/YOUR_USER_NAME/projects
      2. Max: /Users/YOUR_USER_NAME/projects
      3. Windows would looks something like C:\Users\YOUR_USER_NAME\projects

  2. Install Git

  3. Clone repository into your DEVELOPMENT DIRECTORY

    cd /YOUR/DEVELOPMENT/DIRECTORY/
git clone https://github.com/Ecotrust/TEKDB.git

  4. Install Vagrant

  5. Deploy base box with Vagrant

    cd /YOUR/DEVELOPMENT/DIRECTORY/TEKDB/
vagrant up
    1. Wait while your new VM installs all of the required dependencies
    2. if you see an error like: 
          default: cannot touch '/home/ubuntu/.bash_aliases'
    default: : Permission denied
    default: /vagrant/scripts/vagrant_provision_ubuntu.sh: line 43: /home/ubuntu/.bash_aliases: Permission denied
The SSH command responded with a non-zero exit status. Vagrant assumes that this means the command failed. The output for this command should be in the log above. Please read the output to determine what went wrong.
      1. this likely means your vagrant VM is using 'vagrant' for the default user, but all of the code assumes 'ubuntu' as the default user. Fix this by:
        1. vagrant halt
        2. vagrant destroy
        3. Comment out the line in Vagrantfile that reads: config.ssh.username = "ubuntu"
          • vagrant assumes a user of 'vagrant' - the box assigns all passwords and public keys to this user, so we need to log in with this user, create the 'ubuntu' user that we need to finish up, and re-provision from there.
        4. vagrant up
        5. vagrant ssh
        6. sudo cp /home/vagrant/.ssh/authorized_keys /home/ubuntu/.ssh/
        7. exit
        8. Then edit Vagrantfile to include: config.ssh.username = "ubuntu"
        9. vagrant halt
        10. vagrant up --provision

  6. Log in to your new VM and run updates

    vagrant ssh
sudo apt-get update
sudo apt-get upgrade

  7. Update Local Settings

    1. File (same file, just accessed from either the host or the vm):
      1. On Host server: /YOUR/DEVELOPMENT/DIRECTORY/TEKDB/TEKDB/TEKDB/local_settings.py
      2. On Vagrant VM: /usr/local/apps/TEKDB/TEKDB/local_settings.py
    2. Things to change:
      1. DEBUG: add a line that reads DEBUG=True - this is only to be used for development environments, not live servers.
      2. SECRET_KEY: change to a long string of gibberish - you never need to type this in, so the longer and more complex the better
      3. DATABASE_GEOGRAPHY: basic geo settings so the maps know where to focus by default in the admin
        1. For example:
        DATABASE_GEOGRAPHY = {
    'default_lon': -124.2,
    'default_lat': 41.9,
    'default_zoom': 10,
    'map_template': 'gis/admin/ol2osm.html',
}
        1. NOTES:
          1. LAT/LON: the lon and lat presented here are Coordinates (EPSG:4326) for the Northern CA coast. At the time of this writing, the tool only supports coordinates in Web Mercator (EPSG:3857), so the longitude would actually be -13825880.7546 and the latitude is 5146011.678566459.
          2. ZOOM: The higher the number, the more closely zoomed in the map will be by default.
          3. TEMPLATE: The location of the html template for displaying the map in the admin tool for geometry fields.

  8. Activate your python virtual environment

    source /usr/local/apps/TEKDB/env/bin/activate

  9. Run migration

    python /usr/local/apps/TEKDB/manage.py migrate

  10. Run static file collection

    python /usr/local/apps/TEKDB/manage.py collectstatic

  11. Load in the dummy data

    python /usr/local/apps/TEKDB/manage.py loaddata /usr/local/apps/TEKDB/TEKDB/fixtures/all_dummy_data.json

  12. Test your installation with Django’s Dev Server

    python /usr/local/apps/TEKDB/manage.py runserver 0.0.0.0:8000
    1. Open http://localhost:8111
Clone this wiki locally