The virtual disk for my lemmy instance filled up which caused lemmy to throw a lot of errors. I resized the disk and expanded the filesystem but now the pictrs container is constantly restarting.

root@Lemmy:/srv/lemmy# le 
less         lessecho     lessfile     lesskey      lesspipe     let          letsencrypt  lexgrog      
root@Lemmy:/srv/lemmy# ls
leemyalone.org
root@Lemmy:/srv/lemmy# cd leemyalone.org/
root@Lemmy:/srv/lemmy/leemyalone.org# docker-compose ps
          Name                        Command                 State                           Ports                      
-------------------------------------------------------------------------------------------------------------------------
leemyaloneorg_lemmy-ui_1   docker-entrypoint.sh /bin/ ...   Up           1234/tcp                                        
leemyaloneorg_lemmy_1      /app/lemmy                       Up                                                           
leemyaloneorg_pictrs_1     /sbin/tini -- /usr/local/b ...   Restarting                                                   
leemyaloneorg_postfix_1    /root/run                        Up           25/tcp                                          
leemyaloneorg_postgres_1   docker-entrypoint.sh postgres    Up           5432/tcp                                        
leemyaloneorg_proxy_1      /docker-entrypoint.sh ngin ...   Up           80/tcp, 0.0.0.0:3378->8536/tcp,:::3378->8536/tcp
  • matthew@leemyalone.orgOP
    link
    fedilink
    English
    arrow-up
    1
    ·
    1 year ago

    Can’t see pictrs log because it never full starts.

    root@Lemmy:/srv/lemmy/leemyalone.org# docker-compose logs leemyaloneorg_pictures_1
    ERROR: No such service: leemyaloneorg_pictures_1
    root@Lemmy:/srv/lemmy/leemyalone.org#
    
    • Kangie@lemmy.srcfiles.zip
      link
      fedilink
      English
      arrow-up
      2
      ·
      edit-2
      1 year ago

      If the pictrs container doesn’t start check the docker logs.

      journalctl -fexu docker

      It’ll typically tell you why a container isn’t starting, usually a broken bind mount.

      To prevent this from happening again, try migrating to an S3 backend; DigitalOcean have one that’s fixed-price and includes egress, so you can’t accidentally end up with a ridiculous bill one month!

    • john armoredcore @lemmygrad.ml
      link
      fedilink
      English
      arrow-up
      1
      arrow-down
      1
      ·
      edit-2
      1 year ago

      You can still see the logs using docker logs container_name>. To get the container name you can use docker ps -a. It should list the pictrs container there. The container name is usually the last column of the output.

      • matthew@leemyalone.orgOP
        link
        fedilink
        English
        arrow-up
        3
        ·
        edit-2
        1 year ago

        ` ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ BACKTRACE ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

        Run with COLORBT_SHOW_HIDDEN=1 environment variable to disable frame filtering. 2023-08-26T20:46:43.679371Z WARN sled::pagecache::snapshot: corrupt snapshot file found, crc does not match expected Error: 0: Error in database 1: Read corrupted data at file offset None backtrace ()

        Location: src/repo/sled.rs:84

        ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ SPANTRACE ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

        0: pict_rs::repo::sled::build with path=“/mnt/sled-repo” cache_capacity=67108864 export_path=“/mnt/exports” at src/repo/sled.rs:78 1: pict_rs::repo::open with config=Sled(Sled { path: “/mnt/sled-repo”, cache_capacity: 67108864, export_path: “/mnt/exports” }) at src/repo.rs:464

        root@Lemmy:~#`

          • matthew@leemyalone.orgOP
            link
            fedilink
            English
            arrow-up
            3
            ·
            edit-2
            1 year ago

            Is there a way to reset the pictrs DB without affecting the posts, commenst and users DB?

            • john armoredcore @lemmygrad.ml
              link
              fedilink
              English
              arrow-up
              2
              arrow-down
              1
              ·
              1 year ago

              pictrs database is completely separate from lemmy database. If you want you just delete everything in the pictrs volume and start afresh. You will lose all images though.

              • matthew@leemyalone.orgOP
                link
                fedilink
                English
                arrow-up
                2
                ·
                1 year ago

                There’s only two local posts on my instance so i’m not worried about losing thoseabout.

                Will the pictrs from subscribed communities in other instances be restored after a db reset?

                • john armoredcore @lemmygrad.ml
                  link
                  fedilink
                  English
                  arrow-up
                  1
                  arrow-down
                  1
                  ·
                  1 year ago

                  pictrs from subscribed communities in other instances

                  Dont worry about those. Those images are not stored by your pictrs instance. They are directly fetched from the native instances.

              • matthew@leemyalone.orgOP
                link
                fedilink
                English
                arrow-up
                1
                ·
                edit-2
                1 year ago

                OK. I just deleted the pictrs folder from srv/lemmy/leemyalone.org/volumes but I am still having the same issue.

                • john armoredcore @lemmygrad.ml
                  link
                  fedilink
                  English
                  arrow-up
                  3
                  arrow-down
                  1
                  ·
                  1 year ago

                  You’ll have to check again. My guess is though you should not have deleted the pictrs folders. Just its contents. You can recreate the folder again though:

                  mkdir pictrs
                  sudo chown 991:991 pictrs
                  

                  But check the logs first.

                  • matthew@leemyalone.orgOP
                    link
                    fedilink
                    English
                    arrow-up
                    3
                    ·
                    edit-2
                    1 year ago

                    In my previous comment I should have mentioned that I did recreate the pictrs folder. Your instruction to change set the ownership of the pictures folder to 991:991 did the trick.

                    Thank you so much for your help here. Much appreciated.

            • SpecGeo@lemmy.world
              link
              fedilink
              English
              arrow-up
              1
              ·
              1 year ago

              You can try mounting a new folder as pictrs volume. I assume your other data will be safe since it is in the database.