Filter S3 files in bucket by pattern using AWS CLI
Filtering S3 files using a pattern, or finding all files in an S3 bucket that contain a substring can be completed using the AWS CLI.
Filter by S3 object prefix
If the S3 object prefix is predictable, a quick solution is using the native --prefix argument.
1aws s3api list-objects-v2 \
2 --bucket BUCKET_NAME \
3 --prefix PREFIX_STRINGOutput:
1{
2 "Contents":[
3 {
4 "Key":"prefix/",
5 "LastModified":"2021-11-22T07:51:03+00:00",
6 "ETag":"...",
7 "Size":0,
8 "StorageClass":"STANDARD"
9 },
10 {
11 "Key":"prefix/example.txt",
12 "LastModified":"2021-12-12T17:18:32+00:00",
13 "ETag":"...",
14 "Size":1646,
15 "StorageClass":"STANDARD"
16 }
17 ]
18}Find S3 objects containing keyword/substring
To find all S3 object keys that contain a certain substring, the --query argument can be utilized.
1aws s3api list-objects-v2 \
2 --bucket BUCKET_NAME \
3 --query "Contents[?contains(Key, `SEARCH_PATTERN`)]"Output:
1[
2 {
3 "Key":"name-with-substring",
4 "LastModified":"2021-12-12T17:18:32+00:00",
5 "ETag":"...",
6 "Size":1646,
7 "StorageClass":"STANDARD"
8 },
9 {
10 "Key":"another-name-with-substring",
11 "LastModified":"2021-11-22T07:52:45+00:00",
12 "ETag":"...",
13 "Size":1645,
14 "StorageClass":"STANDARD"
15 }
16]