The secrets in your

git repository

06.10.2021

6000 internal Twitch.tv git repositories
were leaked to the public

Inside the repositories researchers found:

194 AWS keys
69 Twilio keys
68 Google API keys
14 Github OAuth keys
4 Stripe keys
1000s of passwords
100s of database connection strings
100s of private keys

What if attackers had access to your internal repositories?

... would you be worried?

Topics

Why are there credentials in git repositories? 🤔
Tools to find credentials 🔎
Mining public git repositories for AWS keys ⛏️
Where else can credentials be found? 🤷
API keys and cloud security ☁️
How can I protect myself? 🛡️

Why are there credentials in git repositories? 🤔

How Secrets are used

Secrets are used to authenticate software against external services.

Secrets are committed by accident

The 4 maturity levels of secret removal

Level 0

Credentials are not deleted

Level 1

A commit deletes the secret from the code. It still persists in the git history.

Still part of the history

Level 2

Git history is rewritten. Still vulnerable to monitoring and advanced repository analysis techniques.

Level 3

Secret is rotated. Can be painful and time consuming.

Did you know...

Secrets not deleted after 24 hours tend to stay forever

Git repositories are everywhere

Configuration issue of the git server could make all repositories public
Local copies are shared
Build server misconfiguration pushes git repositories to artifact stores
Repositories are deployed on a web server

Tools to find credentials 🔎

How do they work?

Heuristics

Pattern matching (regex)
Entropy
Context information

Trufflehog

New Version 3 was released recently
Scans the history of a repository for secrets
Over 700 credential detectors
Support for active verification

Credentials detectors


						idPat  = regexp.MustCompile(`\b((?:AKIA|ABIA|ACCA|ASIA)[0-9A-Z]{16})\b`)
						keyPat = regexp.MustCompile(`\b([A-Za-z0-9+/]{40})\b`)

Active verification


						// REQUEST VALUES.
						method := "GET"
						service := "sts"
						host := "sts.amazonaws.com"
						region := "us-east-1"
						endpoint := "https://sts.amazonaws.com"
						datestamp := time.Now().UTC().Format("20060102")
						amzDate := time.Now().UTC().Format("20060102T150405Z0700")

						req, err := http.NewRequestWithContext(ctx, method, endpoint, nil)
						if err != nil {
							continue
						}

						// TASK 1: CREATE A CANONICAL REQUEST.
						// http://docs.aws.amazon.com/general/latest/gr/sigv4-create-canonical-request.html
						canonicalURI := "/"
						canonicalHeaders := "host:" + host + "\n"
						signedHeaders := "host"
						algorithm := "AWS4-HMAC-SHA256"
						credentialScope := fmt.Sprintf("%s/%s/%s/aws4_request", datestamp, region, service)

						params := req.URL.Query()
						params.Add("Action", "GetCallerIdentity")
						params.Add("Version", "2011-06-15")
						params.Add("X-Amz-Algorithm", algorithm)
						params.Add("X-Amz-Credential", resIdMatch+"/"+credentialScope)
						params.Add("X-Amz-Date", amzDate)
						params.Add("X-Amz-Expires", "30")
						params.Add("X-Amz-SignedHeaders", signedHeaders)

Gitleaks

Alternative to trufflehog
Similar feature set
Many knobs and buttons
Does not offer active verification

Git Hound

Built for bug hunters / Audits
Intensive repository digging
Results need to be reviewed manually

Commercial SaaS provider
Easy integration in build pipelines
Additional Features like Alerting and Dashboards

Mining public git repositories for AWS keys ⛏️

Goals of project

Verify theory that many Developers don't know how to delete accidental secret commits.
Verify theory that there are many active credentials present on public git repositories.
Search for AWS keys.