mongoq: Authentication mechanism not supported

[eduardbosch]

Hi,

I'm trying to use mongoq to browse my backups but I'm having some trouble. I think there's something wrong with mongoq. I hope anyone can help me.

The first problem is that I'm trying to run the command
./mongoq -b=all_backups -p=mongo-rocks -m=/path/to/desired/mountpoint

But mongoq says that flag "m" is unknown, so I cannot specify my S3 replica path.
unknown flag 'm'

If I don't specify the replica path, then mongoq says this:

panic: The authorization mechanism you have provided is not supported. Please use AWS4-HMAC-SHA256.

goroutine 1 [running]:
panic(0x91be00, 0xc8201ce0c0)
    /usr/local/go/src/runtime/panic.go:481 +0x3e6
github.com/facebookgo/rocks-strata/strata/mongo.RunCLI(0x7f8f6c217b28, 0xc8200f2540)
    /home/ubuntu/go_work/src/github.com/facebookgo/rocks-strata/strata/mongo/mongoq.go:484 +0x1a8
main.main()
    /home/ubuntu/go_work/src/github.com/facebookgo/rocks-strata/strata/cmd/mongo/lreplica_drivers/mongoq/main.go:14 +0x78

Thanks for this great tools! I'm using strata for backup/restore and works like a charm 👍

[AGFeldman]

Hmm, using git bisect, the issue with unknown flag 'm' seems to be due to this update to go-flags: jessevdk/go-flags@7f2ab82. We should eventually fix this on our end, but you can try to work around it by using an earlier version of go-flags: cd $GOPATH/src/github.com/jessevdk/go-flags/ ; git checkout 97448c9 and then rebuild mongoq.

Please comment on this issue if authentication is still a problem.

Thanks!

[eduardbosch]

Hi!

Thanks a lot. Flag problem is solved by using earlier version of go-flags.

I still have problems with authentication. I've forgotten to say that I'm using region eu-central-1. The command looks like this

./mongoq -b=all_backups -p=mongo-rocks -p=app-db -R=eu-central-1 -m=/path/to/desired/mountpoint

I think that european servers require to use AWS v4 authentication (http://stackoverflow.com/questions/26533245/the-authorization-mechanism-you-have-provided-is-not-supported-please-use-aws4).

Thanks again!

[eduardbosch]

Hi again,

I've removed the region parameter "-R" and now I can access the extended mogoDB shell.

In the other hand, I think I haven't configured something wrong, because I cannot see my backed up database, but I'll try to set it up again.

Thanks!

[AGFeldman]

Ok. One suggestion is make sure that yas3fs is working by running ls /path/to/desired/mountpoint. You should see a few files and a non-empty metadata subdirectory.

[eduardbosch]

Hi @AGFeldman,

I could get mongoq working by runing yas3fs in a sepparated shell. I didn't undestand that I must have yas3fs running while running mongoq.

I only still have a minor problem, but I tell you to know if it's intended or not.

When I run mongoq, before taking a backup by id with command cb, all logs are bad intended and it's really difficult to read at logs. I show you an example here:

2016/09/22 09:38:17 Checking that database is not in use
                                                        2016/09/22 09:38:17 Read-only restore finished in 1.209899024s
                                                                                                                      2016-09-22T09:38:17.361+0200 I STORAGE  Compression: snappy
      2016-09-22T09:38:17.362+0200 I STORAGE  MaxWriteMBPerSec: 1024
                                                                    2016-09-22T09:38:17.362+0200 I STORAGE  Engine custom option: max_open_files=10
                                                                                                                                                   2016-09-22T09:38:17.362+0200 I STORAGE  Crash safe counters: 0
                                      2016-09-22T09:38:17.365+0200 I ACCESS   Initialized External Auth Session
                                                                                                               2016-09-22T09:38:17.367+0200 I CONTROL  [initandlisten] MongoDB starting : pid=3301 port=27017 dbpath=/tmp/mongoq_938790624 64-bit host=ip-172-31-20-175
                                                                                            2016-09-22T09:38:17.367+0200 I CONTROL  [initandlisten] db version v3.0.12-1.8
                                                                                                                                                                          2016-09-22T09:38:17.367+0200 I CONTROL  [initandlisten] git version: 912604443389db4ed1542d520ac51203b3789e3c
                                                                                                            2016-09-22T09:38:17.367+0200 I CONTROL  [initandlisten] build info: Linux do-ubuntu-trusty-x64-template-8gb-nyc3-01 3.13.0-86-generic #131-Ubuntu SMP Thu May 12 23:33:13 UTC 2016 x86_64 BOOST_LIB_VERSION=1_49
                                                                                                                                                 2016-09-22T09:38:17.367+0200 I CONTROL  [initandlisten] allocator: tcmalloc
                                                 2016-09-22T09:38:17.367+0200 I CONTROL  [initandlisten] options: { net: { port: 27017 }, storage: { dbPath: "/tmp/mongoq_938790624", engine: "rocksdb", rocksdb: { configString: "max_open_files=10" } } }
                                                                                2016-09-22T09:38:17.389+0200 E NETWORK  [initandlisten] listen(): bind() failed errno:98 Address already in use for socket: 0.0.0.0:27017
                                              MONGOQ Trying to start mongod again on another port
                                                                                                 2016-09-22T09:38:17.400+0200 I STORAGE  Compression: snappy
                                                                                                                                                            2016-09-22T09:38:17.400+0200 I STORAGE  MaxWriteMBPerSec: 1024
                                               2016-09-22T09:38:17.400+0200 I STORAGE  Engine custom option: max_open_files=10
                                                                                                                              2016-09-22T09:38:17.400+0200 I STORAGE  Crash safe counters: 0
                 2016-09-22T09:38:17.403+0200 I ACCESS   Initialized External Auth Session
                                                                                          2016-09-22T09:38:17.405+0200 I CONTROL  [initandlisten] MongoDB starting : pid=3303 port=27018 dbpath=/tmp/mongoq_938790624 64-bit host=ip-**********
                                                                       2016-09-22T09:38:17.405+0200 I CONTROL  [initandlisten] db version v3.0.12-1.8
                                                                                                                                                     2016-09-22T09:38:17.405+0200 I CONTROL  [initandlisten] git version: 912604443389db4ed1542d520ac51203b3789e3c
                                                                                       2016-09-22T09:38:17.405+0200 I CONTROL  [initandlisten] build info: Linux do-ubuntu-trusty-x64-template-8gb-nyc3-01 3.13.0-86-generic #131-Ubuntu SMP Thu May 12 23:33:13 UTC 2016 x86_64 BOOST_LIB_VERSION=1_49
                                                                                                                            2016-09-22T09:38:17.405+0200 I CONTROL  [initandlisten] allocator: tcmalloc
                            2016-09-22T09:38:17.405+0200 I CONTROL  [initandlisten] options: { net: { port: 27018 }, storage: { dbPath: "/tmp/mongoq_938790624", engine: "rocksdb", rocksdb: { configString: "max_open_files=10" } } }
                                                           2016-09-22T09:38:19.387+0200 I CONTROL  [initandlisten] 
                                                                                                                   2016-09-22T09:38:19.387+0200 I CONTROL  [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/enabled is 'always'.
                                                                    2016-09-22T09:38:19.387+0200 I CONTROL  [initandlisten] **        We suggest setting it to 'never'
                                                                                                                                                                      2016-09-22T09:38:19.387+0200 I CONTROL  [initandlisten] 
                                                   2016-09-22T09:38:19.388+0200 I CONTROL  [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/defrag is 'always'.
   2016-09-22T09:38:19.388+0200 I CONTROL  [initandlisten] **        We suggest setting it to 'never'
                                                                                                     2016-09-22T09:38:19.388+0200 I CONTROL  [initandlisten] 
                                                                                                                                                             2016-09-22T09:38:19.397+0200 I NETWORK  [initandlisten] waiting for connections on port 27018
                                                                               MongoDB shell version: 3.0.12-1.8
connecting to: 127.0.0.1:27018/test
Server has startup warnings: 
2016-09-22T09:38:19.387+0200 I CONTROL  [initandlisten] 
2016-09-22T09:38:19.387+0200 I CONTROL  [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/enabled is 'always'.
2016-09-22T09:38:19.387+0200 I CONTROL  [initandlisten] **        We suggest setting it to 'never'
2016-09-22T09:38:19.387+0200 I CONTROL  [initandlisten] 
2016-09-22T09:38:19.388+0200 I CONTROL  [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/defrag is 'always'.
2016-09-22T09:38:19.388+0200 I CONTROL  [initandlisten] **        We suggest setting it to 'never'
2016-09-22T09:38:19.388+0200 I CONTROL  [initandlisten]

Thanks a lot for helping me run mongoq! 😃

[AGFeldman]

I'm glad it seems to be working!

That indentation is unfortunate. I don't recall coming across that issue :/