I have tried the docker, ansible, and scratch methods. I have been troubleshooting for a month now. I have gotten nowhere. I need someone to help walk me through how to deploy a lemmy server because the guides are absolute trash.

Please help. I’m wasting money running this VPS and for literally nothing.

Edit: So, I’ve tried the ansible method, but I can’t access my server this way. It just keeps saying “UNREACHABLE”. I have generated a dozen keys, none of them work. I have NO PROBLEMS with ssh in Putty. I can use Putty all day. Putty works fine using my ssh key. Ansible does not. No amount of new keys has made any difference. I have countless keys in my stupid droplet because of this hacky garbage.

  • ZMonster@lemmy.worldOP
    2·
    1 year ago

    I am willing to try any method you prefer. I’ve failed at all of them and have no preference. I don’t care how I get it working, I just need to see something work. Please, please, please help!!!

    • hitagi@ani.socialEnglish
      1·
      1 year ago

      Have you tried Lemmy-Easy-Deploy? This is not the method I used to deploy my own instance but I heard from some that this one is easy to use. It might be worth trying.

      • ZMonster@lemmy.worldOPEnglish
        1·
        1 year ago

        Yes, it worked, but as I’ve told others, the dev did not make it for a production use case. They said they intended it to be used by “people in college that are just testing”. Several features don’t work and are not going to be supported, like SMTP, which I will require. They also said they didn’t intend to have more than one user of the instance… No idea who this usecase is for…

        • hitagi@ani.socialEnglish
          4·
          1 year ago

          The usecase is for people who want a single-user/small instance or those who are completely new to selfhosting. I’m pretty sure SMTP works if you configure an external one (like Mailgun or SendGrid) which is what most people do (including me) since the SMTP config is done via lemmy.hjson (or in the case of Lemmy-Easy-Deploy, the config.env file). IMO, the only thing that makes Lemmy-Easy-Deploy worse for production use is Caddy which performs worse than nginx in some benchmarks. But I don’t think it’s that bad.

          But if you don’t want to use Lemmy-Easy-Deploy, then here’s my own documentation on how I installed Lemmy on my server. It’s different from the standard Lemmy docker or ansible install where I use nginx natively instead of containerized as this fixes my certbot issues. I’m not an expert on any of this but this is how I got it running.

          • ZMonster@lemmy.worldOPEnglish
            1·
            1 year ago

            Thanks for the link. I tried it out and had a problem with the “lemmy.hjson” file. It doesn’t look like I would expect, there is a bunch of non-sense. I did the wget command again and checked “config.hjson” and it is identical. So I think there is something wrong with that file.

            • hitagi@ani.socialEnglish
              1·
              1 year ago

              Whoops. Sorry about that. I used the wrong link. I fixed it and it should pull the correct config.hjson file now. Feel free to try again if you wish.

        • Max-P@lemmy.max-p.meEnglish
          2·
          1 year ago

          Lots of people actually. Part of self-hosting is to be in control of your own stuff. My instance is for exclusive use of myself and my friends for example. That way if lemmy.ml or lemmy.world have problems, mine keeps working and will eventually resync when those come back online. I also have my own emails and my own Matrix and my own IRC bouncer and my own lots of things. That way I’m in full control of my experience, and I own all my data.

          There’s a fair bit of single user lemmy instances out there. It’s a valid use case, just not yours.

  • whenever8186@feddit.uk
    2·
    1 year ago

    Do you have a local device you can try all this on first so you’re not wasting money?

    I agree the guides aren’t great, but they assume you have some experience doing this stuff.

    • ZMonster@lemmy.worldOP
      2·
      1 year ago

      but they assume you have some experience doing this stuff

      I can appreciate that, but I don’t. So, if you are saying that I should just quit because I’m not an experience web developer, then thanks, and no thanks.

      For the better part of the last two decades, once or twice a year I throw a linux distro on a computer to see how it is. And of those several dozen times, EVERY SINGLE TIME, I have encountered an issue that needed troubleshooting and was never resolved. And every single time, I have found myself being advised that “you just have to know how to do this already”. So I get what you are saying, but I don’t. I don’t have experience with this because it always ends up like this. I fucking hate windows, but it works. I can google a problem and find a solution within minutes. Literally any problem has a solution. Not so with linux. Just a lot of people telling me I just need to “know more”.

      • appel@whiskers.bim.boats
        2·
        1 year ago

        I’m sorry you’ve had this experience. It’s not nice to be told that you shouldn’t even try. I think though, that this poster does have a point in that it might be a good idea to stop paying for the hosting of the VPS, practice installing on a local device you have at home, spare laptop etc.(a VM on your own PC would probably work too) and then when you have that going smoothly, try on the VPS again. I haven’t personally used the Ansible install so not sure of it’s quirks, but I’ll try and answer some of your questions.

      • whenever8186@feddit.uk
        2·
        1 year ago

        I am not telling you to quit I’m not telling you to know more. I was merely suggesting that the devs put out the minimal required documentation for an experienced admin to get it up and running, while also suggesting a way to not waste money.

        Sorry it came out wrong.

  • BlackEco@lemmy.blackeco.comEnglish
    1·
    1 year ago

    So, here’s something that might work. I tested it on my local machine, up to Caddy but without HTTPS, but I’m confident it’ll work once deployed on a server.

    Prerequisites:

    • Server with Docker and docker-compose installed
    • Ports 80 and 443 open and directed at your server
    • A domain name pointing to your server

    Setup

    First, create a folder and download the following files:

    Then, generate passwords for PostgreSQL and your admin user, store them somewhere safe.

    Config changes

    lemmy.hjson

    You’ll want to change the admin_username, admin_password and site_name to match your primary user’s credentials and the name you want to give your instance.

    Then, change hostname to match your domain name: if it is sub.domain.tld then it should read hostname: "sub.domain.tld".

    The base config file does not have proper configuration for the database, so you’ll have to edit the database field as follows with the password you previously created:

      database: {
    host: postgres
    database: "lemmy"
    user: "lemmy"
    password: "POSTGRES_PWD" # Change for your password
  }

    Additionally, if you want to send emails for registration confirmation and password resets, add the following before the closing } and change to match your email provider configuration.

      email: {
      # Hostname and port of the smtp server
      smtp_server: "SMTP_SERVER"
      # Login name for smtp server
      smtp_login: "SMTP_LOGIN"
      # Password to login to the smtp server
      smtp_password: "SMTP_PASSWORD"
      # Address to send emails from, eg "noreply@your-instance.com"
      smtp_from_address: "SMTP_LOGIN"
      # Whether or not smtp connections should use tls. Can be none, tls, or starttls
      tls_type: "starttls"
    }

    docker-compose.yml

    By default the compose file is meant to build a development version of Lemmy, we will change this by removing the blocks with build and uncomment those with image. Note: think to update the images to 0.18.2 since it fixes some vulnerabilities.

    Also, since we will use a reverse proxy and I don’t now if your server has a firewall, we should remove the ports blocks which are used to expose the services’ ports on the host.

    Finally, make sure to change the POSTGRES_PASSWORD field to match the PostgreSQL password you set in lemmy.hjson.

    It should look something like that:

    version: "3.7"

x-logging: &default-logging
  driver: "json-file"
  options:
    max-size: "50m"
    max-file: "4"

services:
  proxy:
    image: nginx:1-alpine
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf:ro,Z
    restart: always
    depends_on:
      - pictrs
      - lemmy-ui
    logging: *default-logging

  lemmy:
    # use "image" to pull down an already compiled lemmy. make sure to comment out "build".
    image: dessalines/lemmy:0.18.2
    # platform: linux/x86_64 # no arm64 support. uncomment platform if using m1.
    # use "build" to build your local lemmy server image for development. make sure to comment out "image".
    # run: docker compose up --build

    # this hostname is used in nginx reverse proxy and also for lemmy ui to connect to the backend, do not change
    hostname: lemmy
    restart: always
    environment:
      - RUST_LOG="warn,lemmy_server=debug,lemmy_api=debug,lemmy_api_common=debug,lemmy_api_crud=debug,lemmy_apub=debug,lemmy_db_schema=debug,lemmy_db_views=debug,lemmy_db_views_actor=debug,lemmy_db_views_moderator=debug,lemmy_routes=debug,lemmy_utils=debug,lemmy_websocket=debug"
      - RUST_BACKTRACE=full
    volumes:
      - ./lemmy.hjson:/config/config.hjson:Z
    depends_on:
      - postgres
      - pictrs
    logging: *default-logging

  lemmy-ui:
    # use "image" to pull down an already compiled lemmy-ui. make sure to comment out "build".
    image: dessalines/lemmy-ui:0.18.2
    # platform: linux/x86_64 # no arm64 support. uncomment platform if using m1.
    # use "build" to build your local lemmy ui image for development. make sure to comment out "image".
    # run: docker compose up --build

    # build:
    #   context: ../../lemmy-ui # assuming lemmy-ui is cloned besides lemmy directory
    #   dockerfile: dev.dockerfile
    environment:
      # this needs to match the hostname defined in the lemmy service
      - LEMMY_UI_LEMMY_INTERNAL_HOST=lemmy:8536
      # set the outside hostname here
      - LEMMY_UI_LEMMY_EXTERNAL_HOST=localhost:1236
      - LEMMY_UI_HTTPS=false
      - LEMMY_UI_DEBUG=true
    depends_on:
      - lemmy
    restart: always
    logging: *default-logging
    init: true

  pictrs:
    image: asonix/pictrs:0.4.0-beta.19
    # this needs to match the pictrs url in lemmy.hjson
    hostname: pictrs
    # we can set options to pictrs like this, here we set max. image size and forced format for conversion
    # entrypoint: /sbin/tini -- /usr/local/bin/pict-rs -p /mnt -m 4 --image-format webp
    environment:
      - PICTRS_OPENTELEMETRY_URL=http://otel:4137
      - PICTRS__API_KEY=API_KEY
      - RUST_LOG=debug
      - RUST_BACKTRACE=full
      - PICTRS__MEDIA__VIDEO_CODEC=vp9
      - PICTRS__MEDIA__GIF__MAX_WIDTH=256
      - PICTRS__MEDIA__GIF__MAX_HEIGHT=256
      - PICTRS__MEDIA__GIF__MAX_AREA=65536
      - PICTRS__MEDIA__GIF__MAX_FRAME_COUNT=400
    user: 991:991
    volumes:
      - ./volumes/pictrs:/mnt:Z
    restart: always
    logging: *default-logging

  postgres:
    image: postgres:15-alpine
    # this needs to match the database host in lemmy.hson
    # Tune your settings via
    # https://pgtune.leopard.in.ua/#/
    # You can use this technique to add them here
    # https://stackoverflow.com/a/30850095/1655478
    hostname: postgres
    command:
      [
        "postgres",
        "-c",
        "session_preload_libraries=auto_explain",
        "-c",
        "auto_explain.log_min_duration=5ms",
        "-c",
        "auto_explain.log_analyze=true",
        "-c",
        "track_activity_query_size=1048576",
      ]
    environment:
      - POSTGRES_USER=lemmy
      - POSTGRES_PASSWORD=password # Change with your password
      - POSTGRES_DB=lemmy
    volumes:
      - ./volumes/postgres:/var/lib/postgresql/data:Z
    restart: always
    logging: *default-logging

    Caddy

    For the final touch, we are going to setup Caddy, a reverse proxy with HTTPS support out of the box.

    First, create a file nammed Caddyfile and write the following in it:

    sub.domain.tld {
	reverse_proxy http://proxy:1236
}

    Make sure to match your actual domain name.

    Finally, update the docker-compose.yml file to add the following at the end (make sure that it’s correctly tabulated)

      caddy:
    image: caddy:2.6.4
    restart: unless-stopped
    ports:
      - "80:80"
      - "443:443"
      - "443:443/udp"
    depends_on:
      - proxy
    volumes:
      - ./Caddyfile:/etc/caddy/Caddyfile:ro
      - caddy_data:/data
      - caddy_config:/config
volumes:
  caddy_data:
  caddy_config:

    Launching the instance

    Before starting the stack, we have a few things left to do:

    • Create the folders for pictrs and postgres to store their data: mkdir -p volumes/postgres volumes/pictrs
    • Change the owner of volumes/pictrs: sudo chown -R 991:991 pictrs

    Finally, to start everything: docker compose up -d

  • BlackEco@lemmy.blackeco.comEnglish
    1·
    1 year ago

    Could you tell us what has failed / did not work on your previous attempts? Also, what setup did you use, what reverse proxy you had in front of Lemmy if any, etc.

    • ZMonster@lemmy.worldOPEnglish
      2·
      1 year ago

      Okay, so it looks like nginx is a reverse proxy. That is the step I was on. It doesn’t work. Last thing I did was to modify the docker-compose.yml file and change the port from 80 to 1236. I was told to do that by someone who has been trying to troubleshoot with me for 2 weeks now. It doesn’t say to do that anywhere, but they told me to so I did. Then they said to “add this to my upstream” but I haven’t a fucking clue what that means so I don’t know what to do next. There is a config file in etc/nginx/sites-enabled/nginx.conf that the guide directed me to edit. This is the guide that I was directed to use to set up nginx. I was on the step where it requests the nginx status and mine just says “inactive”. I promise, I am following these guides PRECISELY. I don’t know what I am doing wrong, but all the solutions that I have received are things that the guide DOES NOT say to do. So I don’t know where I am. I am happy to start fresh. You just let me know.

      • zergling_man@lemmy.perthchat.orgEnglish
        1·
        1 year ago

        It doesn’t tell you to actually run nginx, it assumes it will automatically run when installed. I don’t know why, that sounds like dumb behaviour even if it were correct. You are right about the guides being trash.

        I recommend getting used to package manager (apt, dpkg) and system daemon/init system (systemd - accessed via systemctl) and then ignoring that guide. Installing and running nginx isn’t complex enough to warrant a guide; installing packages and running services, in general, are just. Configuring nginx, however… If you know the concepts, it’s pretty easy. The concepts are hard.

        • ZMonster@lemmy.worldOPEnglish
          1·
          1 year ago

          It doesn’t tell you to actually run nginx

          That may be so, but the link that they give references a digital ocean guide that DOES (it’s literally step 2). So, am I just ignoring that part? How do I figure out which parts of a guide should be followed and which ones shouldn’t?

          running nginx isn’t complex enough to warrant a guide

          I haven’t gotten it to work yet so I disagree.

      • BlackEco@lemmy.blackeco.comEnglish
        1·
        1 year ago

        In my opinion its best not to touch the nginx that’s set up by Lemmy and it’s better to have another reverse-proxy in front of it.

        I’ll try to come up with an solution later in the day, gotta do my daily at work.

        • ZMonster@lemmy.worldOPEnglish
          1·
          1 year ago

          No problem at all. Thanks for getting back to me. I really do appreciate it!!! I have a busy day today too but I’ll try to be vigilant about responding.

          • BlackEco@lemmy.blackeco.comEnglish
            1·
            1 year ago

            So, here’s something that might work. I tested it on my local machine, up to Caddy but without HTTPS, but I’m confident it’ll work once deployed on a server.

            Prerequisites:

            • Server with Docker and docker-compose installed
            • Ports 80 and 443 open and directed at your server
            • A domain name pointing to your server

            Setup

            First, create a folder and download the following files:

            Then, generate passwords for PostgreSQL and your admin user, store them somewhere safe.

            Config changes

            lemmy.hjson

            You’ll want to change the admin_username, admin_password and site_name to match your primary user’s credentials and the name you want to give your instance.

            Then, change hostname to match your domain name: if it is sub.domain.tld then it should read hostname: "sub.domain.tld".

            The base config file does not have proper configuration for the database, so you’ll have to edit the database field as follows with the password you previously created:

              database: {
    host: postgres
    database: "lemmy"
    user: "lemmy"
    password: "POSTGRES_PWD" # Change for your password
  }

            Additionally, if you want to send emails for registration confirmation and password resets, add the following before the closing } and change to match your email provider configuration.

              email: {
      # Hostname and port of the smtp server
      smtp_server: "SMTP_SERVER"
      # Login name for smtp server
      smtp_login: "SMTP_LOGIN"
      # Password to login to the smtp server
      smtp_password: "SMTP_PASSWORD"
      # Address to send emails from, eg "noreply@your-instance.com"
      smtp_from_address: "SMTP_LOGIN"
      # Whether or not smtp connections should use tls. Can be none, tls, or starttls
      tls_type: "starttls"
    }

            docker-compose.yml

            By default the compose file is meant to build a development version of Lemmy, we will change this by removing the blocks with build and uncomment those with image. Note: think to update the images to 0.18.2 since it fixes some vulnerabilities.

            Also, since we will use a reverse proxy and I don’t now if your server has a firewall, we should remove the ports blocks which are used to expose the services’ ports on the host.

            Finally, make sure to change the POSTGRES_PASSWORD field to match the PostgreSQL password you set in lemmy.hjson.

            It should look something like that:

            version: "3.7"

x-logging: &default-logging
  driver: "json-file"
  options:
    max-size: "50m"
    max-file: "4"

services:
  proxy:
    image: nginx:1-alpine
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf:ro,Z
    restart: always
    depends_on:
      - pictrs
      - lemmy-ui
    logging: *default-logging

  lemmy:
    # use "image" to pull down an already compiled lemmy. make sure to comment out "build".
    image: dessalines/lemmy:0.18.2
    # platform: linux/x86_64 # no arm64 support. uncomment platform if using m1.
    # use "build" to build your local lemmy server image for development. make sure to comment out "image".
    # run: docker compose up --build

    # this hostname is used in nginx reverse proxy and also for lemmy ui to connect to the backend, do not change
    hostname: lemmy
    restart: always
    environment:
      - RUST_LOG="warn,lemmy_server=debug,lemmy_api=debug,lemmy_api_common=debug,lemmy_api_crud=debug,lemmy_apub=debug,lemmy_db_schema=debug,lemmy_db_views=debug,lemmy_db_views_actor=debug,lemmy_db_views_moderator=debug,lemmy_routes=debug,lemmy_utils=debug,lemmy_websocket=debug"
      - RUST_BACKTRACE=full
    volumes:
      - ./lemmy.hjson:/config/config.hjson:Z
    depends_on:
      - postgres
      - pictrs
    logging: *default-logging

  lemmy-ui:
    # use "image" to pull down an already compiled lemmy-ui. make sure to comment out "build".
    image: dessalines/lemmy-ui:0.18.2
    # platform: linux/x86_64 # no arm64 support. uncomment platform if using m1.
    # use "build" to build your local lemmy ui image for development. make sure to comment out "image".
    # run: docker compose up --build

    # build:
    #   context: ../../lemmy-ui # assuming lemmy-ui is cloned besides lemmy directory
    #   dockerfile: dev.dockerfile
    environment:
      # this needs to match the hostname defined in the lemmy service
      - LEMMY_UI_LEMMY_INTERNAL_HOST=lemmy:8536
      # set the outside hostname here
      - LEMMY_UI_LEMMY_EXTERNAL_HOST=localhost:1236
      - LEMMY_UI_HTTPS=false
      - LEMMY_UI_DEBUG=true
    depends_on:
      - lemmy
    restart: always
    logging: *default-logging
    init: true

  pictrs:
    image: asonix/pictrs:0.4.0-beta.19
    # this needs to match the pictrs url in lemmy.hjson
    hostname: pictrs
    # we can set options to pictrs like this, here we set max. image size and forced format for conversion
    # entrypoint: /sbin/tini -- /usr/local/bin/pict-rs -p /mnt -m 4 --image-format webp
    environment:
      - PICTRS_OPENTELEMETRY_URL=http://otel:4137
      - PICTRS__API_KEY=API_KEY
      - RUST_LOG=debug
      - RUST_BACKTRACE=full
      - PICTRS__MEDIA__VIDEO_CODEC=vp9
      - PICTRS__MEDIA__GIF__MAX_WIDTH=256
      - PICTRS__MEDIA__GIF__MAX_HEIGHT=256
      - PICTRS__MEDIA__GIF__MAX_AREA=65536
      - PICTRS__MEDIA__GIF__MAX_FRAME_COUNT=400
    user: 991:991
    volumes:
      - ./volumes/pictrs:/mnt:Z
    restart: always
    logging: *default-logging

  postgres:
    image: postgres:15-alpine
    # this needs to match the database host in lemmy.hson
    # Tune your settings via
    # https://pgtune.leopard.in.ua/#/
    # You can use this technique to add them here
    # https://stackoverflow.com/a/30850095/1655478
    hostname: postgres
    command:
      [
        "postgres",
        "-c",
        "session_preload_libraries=auto_explain",
        "-c",
        "auto_explain.log_min_duration=5ms",
        "-c",
        "auto_explain.log_analyze=true",
        "-c",
        "track_activity_query_size=1048576",
      ]
    environment:
      - POSTGRES_USER=lemmy
      - POSTGRES_PASSWORD=password # Change with your password
      - POSTGRES_DB=lemmy
    volumes:
      - ./volumes/postgres:/var/lib/postgresql/data:Z
    restart: always
    logging: *default-logging

            Reverse-proxy

            For the final touch, we are going to setup Caddy, a reverse proxy with HTTPS support out of the box. You could use pretty much any reverse proxy you want, but I chose Caddy for its easy setup.

            First, create a file nammed Caddyfile and write the following in it:

            sub.domain.tld {
	reverse_proxy http://proxy:1236
}

            Make sure to match your actual domain name.

            Finally, update the docker-compose.yml file to add the following at the end (make sure that it’s correctly tabulated)

              caddy:
    image: caddy:2.6.4
    restart: unless-stopped
    ports:
      - "80:80"
      - "443:443"
      - "443:443/udp"
    depends_on:
      - proxy
    volumes:
      - ./Caddyfile:/etc/caddy/Caddyfile:ro
      - caddy_data:/data
      - caddy_config:/config
volumes:
  caddy_data:
  caddy_config:

            Launching the instance

            Before starting the stack, we have a few things left to do:

            • Create the folders for pictrs and postgres to store their data: mkdir -p volumes/postgres volumes/pictrs
            • Change the owner of volumes/pictrs: sudo chown -R 991:991 pictrs

            Finally, to start everything: docker compose up -d

          • BlackEco@lemmy.blackeco.comEnglish
            1·
            1 year ago

            So, here’s something that might work. I tested it on my local machine, up to Caddy but without HTTPS, but I’m confident it’ll work once deployed on a server.

            Prerequisites:

            • Server with Docker and docker-compose installed
            • Ports 80 and 443 open and directed at your server
            • A domain name pointing to your server

            Setup

            First, create a folder and download the following files:

            Then, generate passwords for PostgreSQL and your admin user, store them somewhere safe.

            Config changes

            lemmy.hjson

            You’ll want to change the admin_username, admin_password and site_name to match your primary user’s credentials and the name you want to give your instance.

            Then, change hostname to match your domain name: if it is sub.domain.tld then it should read hostname: "sub.domain.tld".

            The base config file does not have proper configuration for the database, so you’ll have to edit the database field as follows with the password you previously created:

              database: {
    host: postgres
    database: "lemmy"
    user: "lemmy"
    password: "POSTGRES_PWD" # Change for your password
  }

            Additionally, if you want to send emails for registration confirmation and password resets, add the following before the closing } and change to match your email provider configuration.

              email: {
      # Hostname and port of the smtp server
      smtp_server: "SMTP_SERVER"
      # Login name for smtp server
      smtp_login: "SMTP_LOGIN"
      # Password to login to the smtp server
      smtp_password: "SMTP_PASSWORD"
      # Address to send emails from, eg "noreply@your-instance.com"
      smtp_from_address: "SMTP_LOGIN"
      # Whether or not smtp connections should use tls. Can be none, tls, or starttls
      tls_type: "starttls"
    }

            docker-compose.yml

            By default the compose file is meant to build a development version of Lemmy, we will change this by removing the blocks with build and uncomment those with image. Note: think to update the images to 0.18.2 since it fixes some vulnerabilities.

            Also, since we will use a reverse proxy and I don’t now if your server has a firewall, we should remove the ports blocks which are used to expose the services’ ports on the host.

            Finally, make sure to change the POSTGRES_PASSWORD field to match the PostgreSQL password you set in lemmy.hjson.

            It should look something like that:

            version: "3.7"

x-logging: &default-logging
  driver: "json-file"
  options:
    max-size: "50m"
    max-file: "4"

services:
  proxy:
    image: nginx:1-alpine
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf:ro,Z
    restart: always
    depends_on:
      - pictrs
      - lemmy-ui
    logging: *default-logging

  lemmy:
    # use "image" to pull down an already compiled lemmy. make sure to comment out "build".
    image: dessalines/lemmy:0.18.2
    # platform: linux/x86_64 # no arm64 support. uncomment platform if using m1.
    # use "build" to build your local lemmy server image for development. make sure to comment out "image".
    # run: docker compose up --build

    # this hostname is used in nginx reverse proxy and also for lemmy ui to connect to the backend, do not change
    hostname: lemmy
    restart: always
    environment:
      - RUST_LOG="warn,lemmy_server=debug,lemmy_api=debug,lemmy_api_common=debug,lemmy_api_crud=debug,lemmy_apub=debug,lemmy_db_schema=debug,lemmy_db_views=debug,lemmy_db_views_actor=debug,lemmy_db_views_moderator=debug,lemmy_routes=debug,lemmy_utils=debug,lemmy_websocket=debug"
      - RUST_BACKTRACE=full
    volumes:
      - ./lemmy.hjson:/config/config.hjson:Z
    depends_on:
      - postgres
      - pictrs
    logging: *default-logging

  lemmy-ui:
    # use "image" to pull down an already compiled lemmy-ui. make sure to comment out "build".
    image: dessalines/lemmy-ui:0.18.2
    # platform: linux/x86_64 # no arm64 support. uncomment platform if using m1.
    # use "build" to build your local lemmy ui image for development. make sure to comment out "image".
    # run: docker compose up --build

    # build:
    #   context: ../../lemmy-ui # assuming lemmy-ui is cloned besides lemmy directory
    #   dockerfile: dev.dockerfile
    environment:
      # this needs to match the hostname defined in the lemmy service
      - LEMMY_UI_LEMMY_INTERNAL_HOST=lemmy:8536
      # set the outside hostname here
      - LEMMY_UI_LEMMY_EXTERNAL_HOST=localhost:1236
      - LEMMY_UI_HTTPS=false
      - LEMMY_UI_DEBUG=true
    depends_on:
      - lemmy
    restart: always
    logging: *default-logging
    init: true

  pictrs:
    image: asonix/pictrs:0.4.0-beta.19
    # this needs to match the pictrs url in lemmy.hjson
    hostname: pictrs
    # we can set options to pictrs like this, here we set max. image size and forced format for conversion
    # entrypoint: /sbin/tini -- /usr/local/bin/pict-rs -p /mnt -m 4 --image-format webp
    environment:
      - PICTRS_OPENTELEMETRY_URL=http://otel:4137
      - PICTRS__API_KEY=API_KEY
      - RUST_LOG=debug
      - RUST_BACKTRACE=full
      - PICTRS__MEDIA__VIDEO_CODEC=vp9
      - PICTRS__MEDIA__GIF__MAX_WIDTH=256
      - PICTRS__MEDIA__GIF__MAX_HEIGHT=256
      - PICTRS__MEDIA__GIF__MAX_AREA=65536
      - PICTRS__MEDIA__GIF__MAX_FRAME_COUNT=400
    user: 991:991
    volumes:
      - ./volumes/pictrs:/mnt:Z
    restart: always
    logging: *default-logging

  postgres:
    image: postgres:15-alpine
    # this needs to match the database host in lemmy.hson
    # Tune your settings via
    # https://pgtune.leopard.in.ua/#/
    # You can use this technique to add them here
    # https://stackoverflow.com/a/30850095/1655478
    hostname: postgres
    command:
      [
        "postgres",
        "-c",
        "session_preload_libraries=auto_explain",
        "-c",
        "auto_explain.log_min_duration=5ms",
        "-c",
        "auto_explain.log_analyze=true",
        "-c",
        "track_activity_query_size=1048576",
      ]
    environment:
      - POSTGRES_USER=lemmy
      - POSTGRES_PASSWORD=password # Change with your password
      - POSTGRES_DB=lemmy
    volumes:
      - ./volumes/postgres:/var/lib/postgresql/data:Z
    restart: always
    logging: *default-logging

            Reverse-proxy

            For the final touch, we are going to setup Caddy, a reverse proxy with HTTPS support out of the box. You could use pretty much any reverse proxy you want, but I chose Caddy for its easy setup.

            First, create a file nammed Caddyfile and write the following in it:

            sub.domain.tld {
	reverse_proxy http://proxy:1236
}

            Make sure to match your actual domain name.

            Finally, update the docker-compose.yml file to add the following at the end (make sure that it’s correctly tabulated)

              caddy:
    image: caddy:2.6.4
    restart: unless-stopped
    ports:
      - "80:80"
      - "443:443"
      - "443:443/udp"
    depends_on:
      - proxy
    volumes:
      - ./Caddyfile:/etc/caddy/Caddyfile:ro
      - caddy_data:/data
      - caddy_config:/config
volumes:
  caddy_data:
  caddy_config:

            Launching the instance

            Before starting the stack, we have a few things left to do:

            • Create the folders for pictrs and postgres to store their data: mkdir -p volumes/postgres volumes/pictrs
            • Change the owner of volumes/pictrs: sudo chown -R 991:991 pictrs

            Finally, to start everything: docker compose up -d

    • ZMonster@lemmy.worldOPEnglish
      1·
      1 year ago

      Okay, I keep getting this question re: “reverse proxys”, but I’m having trouble answering this because the guide does not use that language. There are no steps that direct me to edit, add, modify a “reverse proxy”. So if there is a step in the guide that you know to be a “reverse proxy” then please tell me what it is. I am following the guides EXACTLY. There are a lot of missing steps, but most of these are things that were implied by the guide and not explicitly stated.

      I have tried the docker method. I have gotten close with this, as in, I can use the IP to access the instance, but I have not been able to get the domain to work. My A record is set up correctly. I used lemmy-easy-deploy to get it working the first time and it DID, but that dev does not support instances made for actual deployment so many features just don’t work. It’s not feasible at all. But my domain DID work. So I know my A record is set up correctly.

      I have tried the scratch method. That was a nightmare and nothing worked as expected. I am more than happy to try if you are willing to help me negotiate it.

      I have tried the ansible method. My local PC will not connect to the server with ansible. I have created a dozen new keys and NONE of them work. Ansible WILL NOT CONNECT. I have been using Putty to access the server and it works flawlessly, all day, every day. So I obviously have a working key. Ansible is not working. It just keeps saying “UNREACHABLE”. I have googled this and found no solution that even remotely addressed what I am trying to do.

      I don’t know what to do so you tell me and I’ll fucking do it. Would you like me to start with the docker method? Would you like me to try the ansible method? Would you like me to try the scratch method? You tell me. I don’t know. I don’t care. I just want to get it working.

      • whenever8186@feddit.ukEnglish
        1·
        1 year ago

        There are 2 reverse proxies involved. One is Nginx which is used to front both the Lemmy UI and the Lemmy backend. That’s what the ‘proxy’ container in the docker compose file is for. It seems to be a required component of the application stack as different request types to the same host FQDN are sent to different backends (‘upstreams’ in network speak). You could use Caddy here instead if you wanted, which is the point of this page: https://join-lemmy.org/docs/administration/caddy.html. However, that config doesn’t work for the latest version of Caddy (you’ll get an error about stuff being outside of the site block).

        The other one (could either be Nginx again or Caddy or anything else you want instead) is to front the whole thing and provide TLS termination using Letsencrypt. This bit is explained here: https://join-lemmy.org/docs/administration/install_docker.html#reverse-proxy--webserver

      • BlackEco@lemmy.blackeco.comEnglish
        1·
        1 year ago

        Is the server you’re trying to deploy in a local network? Have you setup your ISP router to route ports 80 and 443 to your server? Or are you using Cloudflare Tunnel?

        As for reverse proxy, it’s usually Caddy or nginx-proxy to get a SSL certificate (for HTTPS) for your service.

        • ZMonster@lemmy.worldOPEnglish
          1·
          1 year ago

          I am using digital ocean.

          Have you setup your ISP router to route ports 80 and 443 to your server?

          I don’t know what that is. What is an ISP router? Is that a config file on my server? You really need to spell it out. I am following the guides. That is what I know. If you have a better guide that uses this verbiage then please send it. I am happy to learn.

          And during the docker method, I was trying to set up nginx and the status is inactive. It will not start.

          Here is the status message:

           Starting A high performance web server and a reverse proxy server...
 nginx[29280]: nginx: [emerg] cannot load certificate "/etc/letsencrypt/live/my_domain_name/fullchain.pem": BIO_new_file() failed (SSL: error:80000002:system>
 nginx[29280]: nginx: configuration file /etc/nginx/nginx.conf test failed
 systemd[1]: nginx.service: Control process exited, code=exited, status=1/FAILURE
 systemd[1]: nginx.service: Failed with result 'exit-code'.
 systemd[1]: Failed to start A high performance web server and a reverse proxy server.
          • zergling_man@lemmy.perthchat.orgEnglish
            1·
            1 year ago

            Look in /etc/nginx/nginx.conf, it probably includes ./sites-available/*.conf, look in there for ssl_certificate(_key) that mentions that fullchain.pem, remove/comment(#) and restart nginx. It may still bitch about not having a cert for ssl, in which case take that out of the listen directives too.

          • BlackEco@lemmy.blackeco.comEnglish
            1·
            1 year ago

            ISP = Internet Service Provider, so your Telco provider. By ISP router I was meaning the box you use at home to connect to the Internet as I presumed you were self-hosting at home. But since you are on Digital Ocean this is irrelevant.

            I’m not familiar with Digital Ocean, do you have access to a standard Linux box with SSH or are you using some sort of Web UI like cPanel to manage it?

            • ZMonster@lemmy.worldOPEnglish
              1·
              1 year ago

              Oh, so my router, 🤣 lol, my bad.

              And yes, I use Putty to access the server currently. root@IP with an RSA key. It works perfectly on Putty. Ansible just will not connect. No idea what to do there. It just says “UNREACHABLE”. I have added and removed a dozen or so keys generated on the WSL Ubuntu Ansible local machine and none of them work.

  • Max-P@lemmy.max-p.me
    1·
    1 year ago

    It seems like you have the keys configured for putty but not whatever SSH client Ansible uses.

    It’s going to sound unhelpful, but have you tried running the Ansible stuff from WSL? This is tested on Linux (and macOS probably), but probably not Windows.

    You may also want to nuke the box of any previous state if you can. The NGINX errors you’re getting are that it expects to find a LetsEncrypt certificate that doesn’t exist yet. Typically you need to start NGINX without it first then certbot will generate it and update the NGINX config to use it.

    Some people have been using Lemmy-Easy-Deploy, it might be worth a try. It’s supposed to be easier and more turnkey. The author is also very open to suggestions and help if the instructions are unclear.

    The docs are definitely geared towards people with existing server admin experience at the moment. There’s talk on Matrix to improve those.

    Please post full logs of what you’re running, it will be easier for people to troubleshoot, including everything before. The extra context can be very helpful to know what Ansible did before and give clues as to what it did previously that might fail. That stuff is a moving target, it gets updated frequently and may fail under some unanticipated conditions.

    • ZMonster@lemmy.worldOP
      1·
      1 year ago

      It’s going to sound unhelpful, but have you tried running the Ansible stuff from WSL? This is tested on Linux (and macOS probably), but probably not Windows. That’s exactly what I’m using, WSL Ubuntu.

      And I’ve tried Lemmy-Easy-Deploy, and it worked fine, but the developer told me that they didn’t intend for this to be used for an actual deployment instance with many users. I plan to scale this as large as I need. They also said that it did not support SMTP and that they had no intention of supporting SMTP. But without SMTP, users won’t be able to confirm emails or reset passwords… So it just won’t work.

      Please post full logs of what you’re running, it will be easier for people to troubleshoot, including everything before. The extra context can be very helpful to know what Ansible did before and give clues as to what it did previously that might fail. That stuff is a moving target, it gets updated frequently and may fail under some unanticipated conditions.

      Good point. I can do that. I have made so many impromptu modifications to my current server that troubleshooting it any further would surely cause errors. So I will blow up the droplet, start fresh, and get it to the first error and post back. Thanks for the advice!

      • Max-P@lemmy.max-p.me
        4·
        1 year ago

        Hmm, I see. In that case I’m afraid you will definitely need to get familiar with a lot of stuff really fast, because there’s a lot of rough edges right now. It’s super beta software after all. Even the admins of lemmy.world and beehaw.org are hitting pretty complex problems that needs a lot of digging. You’ll definitely need to become fairly intimate with how Lemmy works under the hood if you want to scale up to hundreds of users. Especially the pict-rs storage as it grows pretty fast. People have been having to migrate it to object-storage.

        You can always join the install support Matrix room for additional help: https://matrix.to/#/#lemmy-support-new-instance:discuss.online. Or the whole Lemmy Matrix Space at #lemmy-space:matrix.org.

        I don’t want to demotivate you or anything - it’s a great learning experience! But beware you’ll definitely have to troubleshoot and spend some time to keep it running smoothly. It’s far from set and forget at the moment, particularly at scale.

        But one thing at a time, lets see how Ansible does on a blank droplet, and lets figure out how to fix your Ansible problems first before getting into manual installs and scaling up.

        • ZMonster@lemmy.worldOP
          1·
          1 year ago

          I can’t even get that far.

          I used a different PC, installed WSL, created a new rsa key stored at “/home/wsl/.ssh”. Added the public key to my DO droplet. Rebuilt the droplet to ensure that the keys transferred.

          ssh -i /home/wsl/.ssh/id-rsa root@ip

Permission denied (public key)

          every 👏 fucking 👏 time 👏

          I would love to try the ansible method, but apparently I’m the only person to not just instinctively know how to ssh from another pc. I’ve searched for a reference to this and none of them help. Any advice?

          • Max-P@lemmy.max-p.me
            1·
            1 year ago

            I think that’s supposed to be an underscore in id_rsa, not id-rsa, unless you changed it from the default.

            • ZMonster@lemmy.worldOP
              1·
              1 year ago

              I figured out the problem, digital ocean was only applying the top RSA key, which is my Putty key. The keys I added to DO via WSL were not getting added to the authorized key list. So I added my key to the authorized keys file and it worked. Well, it let ansible access the server. I’m having other issues but will work on them later.

  • rglullis@communick.newsEnglish
    1·
    1 year ago

    No to demotivate you from trying it, but if you really want to use your instance instead of working on it, may I suggest you try a managed hosting provider?

    Communick has managed lemmy instances which are affordable, you don’t need to worry about updates or backups and offer a free 14-day trial.

    (Obvious disclosure, I run this service)

  • zergling_man@lemmy.perthchat.org
    1·
    1 year ago

    putty

    That’s your problem. Putty does keys differently to normal ssh.

    ansible

    docker

    .

    So, why can’t you install lemmy? I’m going to assume you’re using ubuntu. If you are willing to swap to arch (which I think DO offers), I recommend it. Get an AUR helper installed (a bit of hassle, I like trizen) and install lemmy from AUR. You’ll need to bump the commit unless you like 0.17.3. I’ll be doing that tonight so I can post relevant bits. If not, post errors.

    Or just keep doing containers so you can have weird and wonderful errors like no dns.