Enable shared Azure and OneLake test resources for maintainers

[kevinjqliu]

@crepererum and I were talking the other day and it might be a good idea to get access to a real azure bucket to test against -- perhaps @kevinjqliu could help facilitate this somehow (I am not sure the best way to integrate real integration tests)

Originally posted by @alamb in #711 (comment)

Context

Disclosure: I am a Microsoft employee and currently work on Microsoft OneLake, our SaaS data lake offering.

I’m happy to help provide real Azure buckets and OneLake accounts for testing against production services. This repo is important to us in Azure Storage and Azure Data, and we want to help ensure compatibility and alignment with the open source ecosystem.

There may also be a possibility of sponsorship through Microsoft Open Source.

I’d love to hear thoughts on the best way to proceed. I can provision accounts and share access with maintainers. I’m also happy to move the conversation to the dev list if that would be better.

[alamb]

What I think is missing from our Azure/OneLake testing strategy is

1. Being able to test a proposed change against a real endpoint to make sure the 1) feature does what is desired, and 2) doesn't break anything else.
2. Ensure that as the backend service continues to evolve, the object_store APIs keep working as designed

We currently use the "Azurite" emulator for this in the integration tests, but that comes with the same caveats as any emulator.

I am not sure that trying to integrate a real azure bucket into the automated CI is a good idea due to the maintenance burden of keeping it working, secret management, etc.

Maybe we could make a semi-automated way to run the tests with some given credentials and then trigger that outside the main repo for particular PRs 🤔