Codebase list gobuster / c3e662c
Merge tag 'upstream/1.2' into kali/master Upstream version 1.2 Sophie Brun 7 years ago
3 changed file(s) with 234 addition(s) and 71 deletion(s). Raw diff Collapse all Expand all
+79
-33
README.md less more
0 Gobuster v1.1 (OJ Reeves @TheColonial)
0 Gobuster v1.2 (OJ Reeves @TheColonial)
11 ======================================
22
3 Alternative directory and file busting tool written in Go. DNS support recently added after inspiration and effort from [Peleus](https://twitter.com/0x42424242).
3 Gobuster is a tool used to brute-force:
4
5 * URIs (directories and files) in web sites.
6 * DNS subdomains (with wildcard support).
47
58 ### Oh dear God.. WHY!?
69
1821
1922 ### But it's shit! And your implementation sucks!
2023
21 Yes, you're probably correct. Feel free to :
24 Yes, you're probably correct. Feel free to:
2225
2326 * Not use it.
2427 * Show me how to do it better.
2629 ### Common Command line options
2730
2831 * `-m <mode>` - which mode to use, either `dir` or `dns` (default: `dir`)
32 * `-q` - disables banner/underline output.
2933 * `-t <threads>` - number of threads to run (default: `10`).
3034 * `-u <url/domain>` - full URL (including scheme), or base domain name.
3135 * `-v` - verbose output (show all results).
3337
3438 ### Command line options for `dns` mode
3539
40 * `-fw` - Force processing of a domain with wildcard DNS.
3641 * `-i` - show all IP addresses for the result.
3742
3843 ### Command line options for `dir` mode
3944
4045 * `-a <user agent string>` - specify a user agent string to send in the request header
4146 * `-c <http cookies>` - use this to specify any cookies that you might need (simulating auth).
47 * `-e` - specify extended mode that renders the full URL.
4248 * `-f` - append `/` for directory brute forces.
4349 * `-l` - show the length of the response.
4450 * `-n` - "no status" mode, disables the output of the result's status code.
4551 * `-p <proxy url>` - specify a proxy to use for all requests (scheme much match the URL scheme)
46 * `-q` - disables banner/underline output.
4752 * `-r` - follow redirects.
4853 * `-s <status codes>` - comma-separated set of the list of status codes to be deemed a "positive" (default: `200,204,301,302,307`).
4954 * `-x <extensions>` - list of extensions to check for, if any.
5560 Since this tool is written in [Go](https://golang.org/) you need install the Go language/compiler/etc. Full details of installation and set up can be found [on the Go language website](https://golang.org/doc/install). Once installed you have two options.
5661
5762 #### Compiling
58 ```
59 gobuster$ go build
60 ```
61 This will create a `gobuster` binary for you.
63 `gobuster` now has external dependencies, and so they need to be pulled in first:
64 ```
65 gobuster $ go get && go build
66 ```
67 This will create a `gobuster` binary for you. If you want to install it in the `$GOPATH/bin` folder you can run:
68 ```
69 gobuster $ go install
70 ```
6271
6372 #### Running as a script
6473 ```
6574 gobuster$ go run main.go <parameters>
6675 ```
6776
77 ### Wordlists via STDIN
78 Wordlists can be piped into `gobuster` via stdin:
79 ```
80 hashcat -a 3 --stdout ?l | gobuster -u https://mysite.com
81 ```
82 Note: If the `-w` option is specified at the same time as piping from STDIN, an error will be shown and the program will terminate.
83
6884 ### Examples
6985
7086 #### `dir` mode
7187
7288 Command line might look like this:
7389 ```
74 $ ./gobuster -u https://mysite.com/path/to/folder -c 'session=123456' -t 50 -w common-files.txt -x .php,.html
90 $ gobuster -u https://mysite.com/path/to/folder -c 'session=123456' -t 50 -w common-files.txt -x .php,.html
7591 ```
7692 Default options looks like this:
7793 ```
78 $ ./gobuster -u http://buffered.io/ -w words.txt
79
80 Gobuster v1.1 OJ Reeves (@TheColonial)
94 $ gobuster -u http://buffered.io/ -w words.txt
95
96 Gobuster v1.2 OJ Reeves (@TheColonial)
8197 =====================================================
8298 [+] Mode : dir
8399 [+] Url/Domain : http://buffered.io/
92108 ```
93109 Default options with status codes disabled looks like this:
94110 ```
95 $ ./gobuster -u http://buffered.io/ -w words.txt -n
96
97 Gobuster v1.1 OJ Reeves (@TheColonial)
111 $ gobuster -u http://buffered.io/ -w words.txt -n
112
113 Gobuster v1.2 OJ Reeves (@TheColonial)
98114 =====================================================
99115 [+] Mode : dir
100116 [+] Url/Domain : http://buffered.io/
110126 ```
111127 Verbose output looks like this:
112128 ```
113 $ ./gobuster -u http://buffered.io/ -w words.txt -v
114
115 Gobuster v1.1 OJ Reeves (@TheColonial)
129 $ gobuster -u http://buffered.io/ -w words.txt -v
130
131 Gobuster v1.2 OJ Reeves (@TheColonial)
116132 =====================================================
117133 [+] Mode : dir
118134 [+] Url/Domain : http://buffered.io/
129145 ```
130146 Example showing content length:
131147 ```
132 $ ./gobuster -u http://buffered.io/ -w words.txt -l
133
134 Gobuster v1.1 OJ Reeves (@TheColonial)
148 $ gobuster -u http://buffered.io/ -w words.txt -l
149
150 Gobuster v1.2 OJ Reeves (@TheColonial)
135151 =====================================================
136152 [+] Mode : dir
137153 [+] Url/Domain : http://buffered.io/
147163 ```
148164 Quiet output, with status disabled and expanded mode looks like this ("grep mode"):
149165 ```
150 $ ./gobuster -u http://buffered.io/ -w words.txt -q -n -e
166 $ gobuster -u http://buffered.io/ -w words.txt -q -n -e
151167 http://buffered.io/posts
152168 http://buffered.io/contact
153169 http://buffered.io/index
157173
158174 Command line might look like this:
159175 ```
160 $ ./gobuster -m dns -u mysite.com -t 50 -w common-names.txt
176 $ gobuster -m dns -u mysite.com -t 50 -w common-names.txt
161177 ```
162178 Normal sample run goes like this:
163179 ```
164 $ ./gobuster -m dns -w subdomains.txt -u google.com
165
166 Gobuster v1.1 OJ Reeves (@TheColonial)
180 $ gobuster -m dns -w subdomains.txt -u google.com
181
182 Gobuster v1.2 OJ Reeves (@TheColonial)
167183 =====================================================
168184 [+] Mode : dns
169185 [+] Url/Domain : google.com
192208 ```
193209 Show IP sample run goes like this:
194210 ```
195 $ ./gobuster -m dns -w subdomains.txt -u google.com -i
196
197 Gobuster v1.1 OJ Reeves (@TheColonial)
211 $ gobuster -m dns -w subdomains.txt -u google.com -i
212
213 Gobuster v1.2 OJ Reeves (@TheColonial)
198214 =====================================================
199215 [+] Mode : dns
200216 [+] Url/Domain : google.com
224240 ```
225241 Base domain validation warning when the base domain fails to resolve. This is a warning rather than a failure in case the user fat-fingers while typing the domain.
226242 ```
227 $ ./gobuster -m dns -w subdomains.txt -u yp.to -i
228
229 Gobuster v1.1 OJ Reeves (@TheColonial)
243 $ gobuster -m dns -w subdomains.txt -u yp.to -i
244
245 Gobuster v1.2 OJ Reeves (@TheColonial)
230246 =====================================================
231247 [+] Mode : dns
232248 [+] Url/Domain : yp.to
233249 [+] Threads : 10
234250 [+] Wordlist : /tmp/test.txt
235251 =====================================================
236 [!] Unable to validate base domain: yp.to
252 [-] Unable to validate base domain: yp.to
237253 Found: cr.yp.to [131.155.70.11, 131.155.70.13]
238254 =====================================================
239255 ```
256 Wildcard DNS is also detected properly:
257 ```
258 $ gobuster -w subdomainsbig.txt -u doesntexist.com -m dns
259
260 Gobuster v1.2 OJ Reeves (@TheColonial)
261 =====================================================
262 [+] Mode : dns
263 [+] Url/Domain : doesntexist.com
264 [+] Threads : 10
265 [+] Wordlist : subdomainsbig.txt
266 =====================================================
267 [-] Wildcard DNS found. IP address(es): 123.123.123.123
268 [-] To force processing of Wildcard DNS, specify the '-fw' switch.
269 =====================================================
270 ```
271 If the user wants to force processing of a domain that has wildcard entries, use `-fw`:
272 ```
273 $ gobuster -w subdomainsbig.txt -u doesntexist.com -m dns -fw
274
275 Gobuster v1.2 OJ Reeves (@TheColonial)
276 =====================================================
277 [+] Mode : dns
278 [+] Url/Domain : doesntexist.com
279 [+] Threads : 10
280 [+] Wordlist : subdomainsbig.txt
281 =====================================================
282 [-] Wildcard DNS found. IP address(es): 123.123.123.123
283 Found: email.doesntexist.com
284 ^C[!] Keyboard interrupt detected, terminating.
285 =====================================================
240286
241287 ### License
242288
+2
-1
THANKS less more
33 @justinsteven - HTTP basic auth support
44 @kevinnz - custom user agent support
55 @viaMorgoth - base domain validation for DNS mode
6 @UID1K - DNS wildcard check support
6 @UID1K - initial DNS wildcard check support
7 @Ne0nd0g - STDIN support for wordlists
+153
-37
main.go less more
2020 "crypto/tls"
2121 "flag"
2222 "fmt"
23 "github.com/satori/go.uuid"
2324 "golang.org/x/crypto/ssh/terminal"
2425 "io/ioutil"
2526 "net"
2627 "net/http"
2728 "net/url"
2829 "os"
30 "os/signal"
2931 "strconv"
3032 "strings"
3133 "sync"
4648 type ProcessorFunc func(s *State, entity string, resultChan chan<- Result)
4749 type SetupFunc func(s *State) bool
4850
49 // Shim type for "set"
51 // Shim type for "set" containing ints
5052 type IntSet struct {
5153 set map[int]bool
54 }
55
56 // Shim type for "set" containing strings
57 type StringSet struct {
58 set map[string]bool
5259 }
5360
5461 // Contains State that are read in from the command
7784 Username string
7885 Verbose bool
7986 Wordlist string
87 IsWildcard bool
88 WildcardForced bool
89 WildcardIps StringSet
90 SignalChan chan os.Signal
91 Terminate bool
92 StdIn bool
8093 }
8194
8295 type RedirectHandler struct {
8699
87100 type RedirectError struct {
88101 StatusCode int
102 }
103
104 // Add an element to a set
105 func (set *StringSet) Add(s string) bool {
106 _, found := set.set[s]
107 set.set[s] = true
108 return !found
109 }
110
111 // Add a list of elements to a set
112 func (set *StringSet) AddRange(ss []string) {
113 for _, s := range ss {
114 set.set[s] = true
115 }
116 }
117
118 // Test if an element is in a set
119 func (set *StringSet) Contains(s string) bool {
120 _, found := set.set[s]
121 return found
122 }
123
124 // Check if any of the elements exist
125 func (set *StringSet) ContainsAny(ss []string) bool {
126 for _, s := range ss {
127 if set.set[s] {
128 return true
129 }
130 }
131 return false
132 }
133
134 // Stringify the set
135 func (set *StringSet) Stringify() string {
136 values := []string{}
137 for s, _ := range set.set {
138 values = append(values, s)
139 }
140 return strings.Join(values, ",")
89141 }
90142
91143 // Add an element to a set
174226 var proxy string
175227 valid := true
176228
177 s := State{StatusCodes: IntSet{set: map[int]bool{}}}
229 s := State{
230 StatusCodes: IntSet{set: map[int]bool{}},
231 WildcardIps: StringSet{set: map[string]bool{}},
232 IsWildcard: false,
233 StdIn: false,
234 }
178235
179236 // Set up the variables we're interested in parsing.
180237 flag.IntVar(&s.Threads, "t", 10, "Number of concurrent threads")
196253 flag.BoolVar(&s.NoStatus, "n", false, "Don't print status codes")
197254 flag.BoolVar(&s.IncludeLength, "l", false, "Include the length of the body in the output (dir mode only)")
198255 flag.BoolVar(&s.UseSlash, "f", false, "Append a forward-slash to each directory request (dir mode only)")
256 flag.BoolVar(&s.WildcardForced, "fw", false, "Force continued operation when wildcard found (dns mode only)")
199257
200258 flag.Parse()
201259
211269 s.Processor = ProcessDnsEntry
212270 s.Setup = SetupDns
213271 default:
214 fmt.Println("Mode (-m): Invalid value:", s.Mode)
272 fmt.Println("[!] Mode (-m): Invalid value:", s.Mode)
215273 valid = false
216274 }
217275
218276 if s.Threads < 0 {
219 fmt.Println("Threads (-t): Invalid value:", s.Threads)
277 fmt.Println("[!] Threads (-t): Invalid value:", s.Threads)
220278 valid = false
221279 }
222280
223 if s.Wordlist == "" {
224 fmt.Println("WordList (-w): Must be specified")
281 stdin, err := os.Stdin.Stat()
282 if err != nil {
283 fmt.Println("[!] Unable to stat stdin, falling back to wordlist file.")
284 } else if (stdin.Mode()&os.ModeCharDevice) == 0 && stdin.Size() > 0 {
285 s.StdIn = true
286 }
287
288 if !s.StdIn {
289 if s.Wordlist == "" {
290 fmt.Println("[!] WordList (-w): Must be specified")
291 valid = false
292 } else if _, err := os.Stat(s.Wordlist); os.IsNotExist(err) {
293 fmt.Println("[!] Wordlist (-w): File does not exist:", s.Wordlist)
294 valid = false
295 }
296 } else if s.Wordlist != "" {
297 fmt.Println("[!] Wordlist (-w) specified with pipe from stdin. Can't have both!")
225298 valid = false
226 } else if _, err := os.Stat(s.Wordlist); os.IsNotExist(err) {
227 fmt.Println("Wordlist (-w): File does not exist:", s.Wordlist)
228 valid = false
229299 }
230300
231301 if s.Url == "" {
232 fmt.Println("Url/Domain (-u): Must be specified")
302 fmt.Println("[!] Url/Domain (-u): Must be specified")
233303 valid = false
234304 }
235305
242312 s.Url = "http://" + s.Url
243313 }
244314
245 // extensions are comma seaprated
315 // extensions are comma separated
246316 if extensions != "" {
247317 s.Extensions = strings.Split(extensions, ",")
248318 for i := range s.Extensions {
252322 }
253323 }
254324
255 // status codes are comma seaprated
325 // status codes are comma separated
256326 if codes != "" {
257327 for _, c := range strings.Split(codes, ",") {
258328 i, err := strconv.Atoi(c)
259329 if err != nil {
260 panic("Invalid status code given")
330 fmt.Println("[!] Invalid status code given: ", c)
331 valid = false
332 } else {
333 s.StatusCodes.Add(i)
261334 }
262 s.StatusCodes.Add(i)
263335 }
264336 }
265337
275347 if err == nil {
276348 s.Password = string(passBytes)
277349 } else {
278 panic("Auth username given but reading of password failed")
350 fmt.Println("[!] Auth username given but reading of password failed")
351 valid = false
279352 }
280353 }
281354
286359 if proxy != "" {
287360 proxyUrl, err := url.Parse(proxy)
288361 if err != nil {
289 panic("Proxy URL is invalid")
362 panic("[!] Proxy URL is invalid")
290363 }
291364 s.ProxyUrl = proxyUrl
292365 proxyUrlFunc = http.ProxyURL(s.ProxyUrl)
308381 fmt.Println("[-] Unable to connect:", s.Url)
309382 valid = false
310383 }
384 } else {
385 Ruler(&s)
311386 }
312387 }
313388
321396 // Process the busting of the website with the given
322397 // set of settings from the command line.
323398 func Process(s *State) {
324 wordlist, err := os.Open(s.Wordlist)
325 if err != nil {
326 panic("Failed to open wordlist")
327 }
328399
329400 ShowConfig(s)
330401
332403 Ruler(s)
333404 return
334405 }
406
407 PrepareSignalHandler(s)
335408
336409 // channels used for comms
337410 wordChan := make(chan string, s.Threads)
375448 printerGroup.Done()
376449 }()
377450
378 defer wordlist.Close()
379
380 // Lazy reading of the wordlist line by line
381 scanner := bufio.NewScanner(wordlist)
451 var scanner *bufio.Scanner
452
453 if s.StdIn {
454 // Read directly from stdin
455 scanner = bufio.NewScanner(os.Stdin)
456 } else {
457 // Pull content from the wordlist
458 wordlist, err := os.Open(s.Wordlist)
459 if err != nil {
460 panic("Failed to open wordlist")
461 }
462 defer wordlist.Close()
463
464 // Lazy reading of the wordlist line by line
465 scanner = bufio.NewScanner(wordlist)
466 }
467
382468 for scanner.Scan() {
469 if s.Terminate {
470 break
471 }
383472 word := strings.TrimSpace(scanner.Text())
384473
385474 // Skip "comment" (starts with #), as well as empty lines
397486
398487 func SetupDns(s *State) bool {
399488 // Resolve a subdomain that probably shouldn't exist
400 _, err := net.LookupHost("bba1b18d-50f8-4f1d-8295-c861445ed7f5." + s.Url)
489 guid := uuid.NewV4()
490 wildcardIps, err := net.LookupHost(fmt.Sprintf("%s.%s", guid, s.Url))
401491 if err == nil {
402 fmt.Println("[-] Wildcard DNS found.")
403 return false
492 s.IsWildcard = true
493 s.WildcardIps.AddRange(wildcardIps)
494 fmt.Println("[-] Wildcard DNS found. IP address(es): ", s.WildcardIps.Stringify())
495 if !s.WildcardForced {
496 fmt.Println("[-] To force processing of Wildcard DNS, specify the '-fw' switch.")
497 }
498 return s.WildcardForced
404499 }
405500
406501 if !s.Quiet {
408503 _, err = net.LookupHost(s.Url)
409504 if err != nil {
410505 // Not an error, just a warning. Eg. `yp.to` doesn't resolve, but `cr.py.to` does!
411 fmt.Println("[!] Unable to validate base domain:", s.Url)
506 fmt.Println("[-] Unable to validate base domain:", s.Url)
412507 }
413508 }
414509
424519 ips, err := net.LookupHost(subdomain)
425520
426521 if err == nil {
427 result := Result{
428 Entity: subdomain,
429 }
430 if s.ShowIPs {
431 result.Extra = strings.Join(ips, ", ")
432 }
433 resultChan <- result
522 if !s.IsWildcard || !s.WildcardIps.ContainsAny(ips) {
523 result := Result{
524 Entity: subdomain,
525 }
526 if s.ShowIPs {
527 result.Extra = strings.Join(ips, ", ")
528 }
529 resultChan <- result
530 }
434531 } else if s.Verbose {
435532 result := Result{
436533 Entity: subdomain,
513610 }
514611 }
515612
613 func PrepareSignalHandler(s *State) {
614 s.SignalChan = make(chan os.Signal, 1)
615 signal.Notify(s.SignalChan, os.Interrupt)
616 go func() {
617 for _ = range s.SignalChan {
618 // caught CTRL+C
619 if !s.Quiet {
620 fmt.Println("[!] Keyboard interrupt detected, terminating.")
621 s.Terminate = true
622 }
623 }
624 }()
625 }
626
516627 func (e *RedirectError) Error() string {
517628 return fmt.Sprintf("Redirect code: %d", e.StatusCode)
518629 }
548659 }
549660
550661 fmt.Println("")
551 fmt.Println("Gobuster v1.1 OJ Reeves (@TheColonial)")
662 fmt.Println("Gobuster v1.2 OJ Reeves (@TheColonial)")
552663 Ruler(state)
553664 }
554665
561672 fmt.Printf("[+] Mode : %s\n", state.Mode)
562673 fmt.Printf("[+] Url/Domain : %s\n", state.Url)
563674 fmt.Printf("[+] Threads : %d\n", state.Threads)
564 fmt.Printf("[+] Wordlist : %s\n", state.Wordlist)
675
676 wordlist := "stdin (pipe)"
677 if !state.StdIn {
678 wordlist = state.Wordlist
679 }
680 fmt.Printf("[+] Wordlist : %s\n", wordlist)
565681
566682 if state.Mode == "dir" {
567683 fmt.Printf("[+] Status codes : %s\n", state.StatusCodes.Stringify())