Download RDS DB logs through AWS CLI

If you need a quicker way to download RDS DB logs, you can do so through AWS CLI (command line interface).  

Although the AWS console also allows downloading these logs, there are disadvantages to this.  Since the console is a graphical user interface, you must click/select the RDS instance first, click on the Monitoring tab of the instance, scroll down to the list of instance logs, and even sort the list of logs in descending order to get to the most recent one -- if you are in the middle of an incident and need to look at the logs to diagnose the issue.  Then you click on the selected log, click the download button, which will bring up a page where you can right-click the selected log using  "save as".  In short, too many mouse clicks before you can get to the needed logs.

Another disadvantage is that you are only allowed to download the log files as they are.  So if the log file is large, you cross your fingers that the network connection will hold steady and won't disconnect in the middle of the download.  If that happens, you have to resume the download and start again.

For faster and more reliable download of RDS DB log files, you can use AWS CLIs instead.  Here is how it is done in Linux.  The same CLIs are also available for Windows.


First, you run the describe-db-log-files command and combine it with the grep Linux OS command to list the existing log files.  You must supply the value for the db-instance-identifier parameter, in this example it is dba-works-01. 

dbaworks@dbasrv1:~$ aws rds describe-db-log-files --db-instance-identifier dba-works-01 | grep "LogFileName"

This will return the list of RDS log files for the dba-works-01 instance.


            "LogFileName": "error/postgresql.log.2021-05-16-12",

            "LogFileName": "error/postgresql.log.2021-05-17-00",

            "LogFileName": "error/postgresql.log.2021-05-17-12",

            "LogFileName": "error/postgresql.log.2021-05-18-00",

            "LogFileName": "error/postgresql.log.2021-05-18-12",

            "LogFileName": "error/postgresql.log.2021-05-19-00",

            "LogFileName": "error/postgresql.log.2021-05-19-12",


Next, run the download-db-log-file-portion  command to begin downloading the selected log file. Again, you should supply the value for the db-instance-identifier parameter, in this example it is dba-works-01, the value for the log-file-name parameter, in this case it is "error/postgresql.log.2021-05-19-12". The next parameter -- starting-token -- allows you to specify where to start paginating the log file.  In this example the value of 0 is used, which means to download the entire file.  The last parameter is output and we use text to specify that the log file should be formatted as text when it is downloaded.  Lastly, we use the Linux pipe to save the results to a local file name dba-works-01-postgresql.log.2021-05-19-12.log


dbaworks@dbasrv1:~$ aws rds download-db-log-file-portion --db-instance-identifier dba-works-01 --log-file-name "error/postgresql.log.2021-05-19-12" --starting-token 0 --output text > dba-works-01-postgresql.log.2021-05-19-12.log


You can also download only the latest portion of the log file by using a simplified command, skipping the -- starting-token parameter, see below: 


dbaworks@dbasrv1:~$ aws rds download-db-log-file-portion --db-instance-identifier dba-works-01 --log-file-name "error/postgresql.log.2021-05-19-12" --output text > dba-works-01-postgresql.log.2021-05-19-12.log

 

The  download-db-log-file-portion command is a nifty tool.  You can run the command with the help option to get more help or information on the other different ways to use this command for your convenience.


aws rds download-db-log-file-portion --db-instance-identifier help



Want to learn more about AWS Command Line Interface (CLI)?  Update your knowledge with this book below:


Labels: , , , ,