[google-auth-library] AWS WIF authentication fails when IMDSv1 is disabled despite imdsv2_session_token_url being configured

[acardote]

Environment

• Node.js version: 22.12.0
• google-auth-library version: 10.5.0 (via @google-cloud/bigquery 8.0.0)
• AWS Region: us-east-1
• EC2 Instance: Amazon Linux 2023 with IMDSv2 required (IMDSv1 disabled)

Description

When using Workload Identity Federation (WIF) to authenticate from AWS EC2 to GCP BigQuery, the library fails to obtain AWS credentials when IMDSv1 is disabled on the EC2 instance, even though the imdsv2_session_token_url is properly configured in the external account credentials file.

The same credentials configuration works correctly with the gcloud CLI (which uses the Python google-auth library), indicating the issue is specific to the Node.js implementation.

Steps to Reproduce

1. Configure an EC2 instance with IMDSv2 required (IMDSv1 disabled):

   aws ec2 modify-instance-metadata-options \
     --instance-id i-xxxxx \
     --http-tokens required \
     --http-endpoint enabled

2. Create an external account credentials file with imdsv2_session_token_url:

   {
     "type": "external_account",
     "audience": "//iam.googleapis.com/projects/PROJECT_NUMBER/locations/global/workloadIdentityPools/POOL_ID/providers/PROVIDER_ID",
     "subject_token_type": "urn:ietf:params:aws:token-type:aws4_request",
     "token_url": "https://sts.googleapis.com/v1/token",
     "credential_source": {
       "environment_id": "aws1",
       "region_url": "http://169.254.169.254/latest/meta-data/placement/availability-zone",
       "url": "http://169.254.169.254/latest/meta-data/iam/security-credentials",
       "regional_cred_verification_url": "https://sts.{region}.amazonaws.com?Action=GetCallerIdentity&Version=2011-06-15",
       "imdsv2_session_token_url": "http://169.254.169.254/latest/api/token"
     },
     "service_account_impersonation_url": "https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/SERVICE_ACCOUNT@PROJECT.iam.gserviceaccount.com:generateAccessToken"
   }

3. Set the credentials path and attempt to use BigQuery:

   import { BigQuery } from '@google-cloud/bigquery';
   
   const bigquery = new BigQuery({
     projectId: 'my-project',
     keyFilename: '/path/to/wif-credentials.json'
   });
   
   const [rows] = await bigquery.query('SELECT 1');

Expected Behavior

The library should:

1. Detect that imdsv2_session_token_url is configured
2. Make a PUT request to obtain an IMDSv2 session token
3. Use that token in subsequent metadata requests
4. Successfully authenticate to GCP

Actual Behavior

The library fails with:

Error: The Security Token included in the request is invalid.
    at AwsClient.retrieveSubjectToken

The underlying error from AWS STS is InvalidClientTokenId, indicating that the library is not properly obtaining AWS credentials via IMDSv2.

Verification

IMDSv1 is actually disabled:

$ curl http://169.254.169.254/latest/meta-data/iam/security-credentials/
# Returns 401 Unauthorized

IMDSv2 works correctly:

$ TOKEN=$(curl -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600")
$ curl -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/iam/security-credentials/
# Returns the IAM role name

gcloud CLI works with the same credentials:

$ export GOOGLE_APPLICATION_CREDENTIALS=/path/to/wif-credentials.json
$ gcloud auth application-default print-access-token
# Successfully prints access token

This confirms:

• The WIF configuration is correct
• The EC2 IAM role has proper permissions
• The GCP service account and WIF pool are properly configured
• Only the Node.js library fails

Workaround

We implemented a workaround by manually fetching AWS credentials via IMDSv2 and setting them as environment variables before creating the BigQuery client:

async function setupAwsCredentialsFromMetadata() {
  // Get IMDSv2 session token
  const token = await fetchImdsv2Token();
  
  // Get IAM role name
  const roleName = await fetchIamRoleName(token);
  
  // Get credentials
  const creds = await fetchAwsCredentials(token, roleName);
  
  // Set environment variables
  process.env.AWS_ACCESS_KEY_ID = creds.AccessKeyId;
  process.env.AWS_SECRET_ACCESS_KEY = creds.SecretAccessKey;
  process.env.AWS_SESSION_TOKEN = creds.Token;
  process.env.AWS_REGION = await fetchAwsRegion(token);
}

// Call before creating BigQuery client
await setupAwsCredentialsFromMetadata();
const bigquery = new BigQuery({ projectId, keyFilename: credentialsPath });

This works because the library will use environment variables when available, bypassing the broken IMDS fetching logic.

Root Cause Hypothesis

Looking at the AwsClient implementation, it appears the library may not be correctly implementing the IMDSv2 flow even when imdsv2_session_token_url is provided. Possible issues:

1. The IMDSv2 token fetch might not be using the correct HTTP method (PUT vs GET)
2. The token might not be properly passed in subsequent metadata requests
3. There might be error handling that falls back to IMDSv1 silently and fails

Related Issues

• Workload Identity with AWS & IMDSv2 use expired token google-auth-library-nodejs#1755 - Mentions IMDSv2 support was added, but may not be complete
• Code relying on AWS IMDSv1 blocking cloud env hardening 😓  google-auth-library-python#1314 - Python library had similar issues

Additional Context

AWS has been pushing IMDSv2 as the security best practice and many organizations are now requiring IMDSv2-only configurations. This makes proper IMDSv2 support critical for production deployments using WIF.

[danieljbruce]

Looks like a problem with the AwsClient. This is not a client library issue is it?

[acardote]

Hi @danieljbruce, thanks for the quick response!

Yes, this is specifically about the AwsClient class in google-auth-library (now part of this monorepo under packages/google-auth-library-nodejs). The AwsClient is the external account credential provider that handles AWS WIF authentication, and it's the component that fails to properly use IMDSv2 when imdsv2_session_token_url is configured in the credential source.

The issue is in how AwsClient.retrieveSubjectToken() fetches AWS credentials from the EC2 instance metadata service. When IMDSv1 is disabled on the instance, the library should use the IMDSv2 flow (PUT request for session token, then use that token in subsequent metadata requests), but it appears to either skip this flow or not implement it correctly, resulting in an InvalidClientTokenId error from AWS STS.

For reference, the AwsClient source was previously at src/auth/awsclient.ts in the original google-auth-library-nodejs repo. The Python equivalent (google.auth.aws) handles this correctly, which is why gcloud CLI works with the same credentials file.

Let me know if this should be tracked elsewhere — happy to re-file if there's a more appropriate location.