do not symlink /run/secrets

Andrew Costa
10 days ago
Working on upgrading to Grouper 2.6.16.2 but we are getting a task failure when trying to deploy. dockerd: time=“2022-10-07T16:13:02.534484050-05:00” level=error msg=“fatal task error” error=“task: non-zero exit (1)” module=node/agent/taskmanager node.id=gt09uw0zp9kagxi1i2aw4e6eb service.id=0ulaa44m7rvfg3hak15uvfvzk task.id=guafz8alfgbjw9c4lzdo83pfo

Andrew Costa
10 days ago
We are doing some caching in the docker file and copying container files to other directories

Andrew Costa
10 days ago
RUN mkdir -p /var/grouper/cache \
&& chgrp tomcat /var/grouper/cache \
&& chmod g+rw /var/grouper/cache

COPY container_files/grouper/ /opt/grouper/
COPY container_files/httpd /etc/httpd/conf.d/

Chris Hyzer
10 days ago
now the user group is tomcat:root, so maybe chgrp root? and maybe call this at end?
/opt/container_files/docker-build-bin/containerDockerfileInstallPermissions.sh tomcat root

Andrew Costa
10 days ago
I will give that a shot on Monday morning. Thanks Chris!

Chris Hyzer
6 days ago
did it work?

Andrew Costa
6 days ago
Still getting the same error. We are at a conference today and tomorrow but Ryan and I will try and figure it out on Thursday

Chris Hyzer
5 days ago
difficult to tell since the error is really shown right?

Andrew Costa
4 days ago
still getting the same error and yes it is difficult to tell since the actual error really isn’t shown

Andrew Costa
4 days ago
I worked with Ryan and we got it working

Chris Hyzer
4 days ago
what was the issue?

Andrew Costa
4 days ago
It was a permissions issue and part of the fix is using ENV GROUPER_CHOWN_DIRS=false that Liam mentioned. It has to do with our docker secrets and moving forward we will have to most likely re-think things and have a discussion about moving our config to the DB

Chris Hyzer
3 days ago
where are the secrets on the file system? can you copy them somewhere else (not in the grouper dir), or can you use slashRoot?

Andrew Costa
3 days ago
They are located in /opt/grouper/grouperwebapp/web-inf/classes/ and symlinked to /run/secrets/

Andrew Costa
3 days ago
The issue is that when deploying it is trying to change permissions on that directory I believe but it is read only which really isn’t an issue but the exit code 1 halts the deployment with a fatal task error

Andrew Costa
3 days ago
So we are using ENV GROUPER_CHOWN_DIRS=false and RUN chown tomcat:root $(find /home/tomcat /opt/container_files /opt/grouper /opt/tier /opt/tier-support /opt/tomee /etc/httpd/conf /home/tomcat /usr/local/bin /etc/httpd/conf.d /usr/lib/jvm/java/jre/lib/security/cacerts -path /opt/grouper/slashRoot -prune -o -path /opt/grouper/logs -prune -o ! -user tomcat -print) at the end of our dockerfile to remedy the issue

Chris Hyzer
1 day ago
@John Gasper (Yale)

@gettes

@chubing
can we copy files from run secrets to the destination instead of symlink? I think that would fix the issue with ownership and permissions, right?
image.png

image.png

Michael Gettes
22 hours ago
I don’t believe you want to attempt to alter/copy /run/secrets files. I am configured to use these files directly. There are cases where the secrets can be changed on the fly and the app should be able to use the changes without restart.

Chris Hyzer
20 hours ago
then why do they need to be in the grouper folder? can you just refer to /run/secrets and we not symlink them?

Michael Gettes
20 hours ago
That's what I do. I don't see a need for them to be in the grouper folder.

Chris Hyzer
20 hours ago
then i propose we make that change, we could have a param for backwards compatibility but by default just refer to them in /run/secrets and no symlinks?
:+1:
1

Chris Hyzer
20 hours ago
or just people change their configs on upgrade step?
:+1:
1

John Gasper
21 minutes ago
I’d strongly encourage not copying them out of /run/secrets. Swarm for sure… and maybe other orchestrators… mount those secrets using a memory backed file system, meaning the secrets never are saved to disk. If someone exports the container secrets in this volume don’t get exported. See