50000 个并行请求的点击 URL

Question

我正在尝试访问任何示例 url 来并行 50000 个请求。但我收到各种错误。有人可以让我知道我做错了什么吗？或者应该采取什么策略？

func MakeRequests(url string, ch chan<- int, wg *sync.WaitGroup) {

    time.Sleep(time.Duration(rand.Intn(10)) * time.Millisecond)

    resp, err := http.Get(url)

    if err != nil {

        fmt.Println(err)

    }

    ch <- resp.StatusCode

    wg.Done()

}

func main() {

    var waitGroup sync.WaitGroup 

    count := 0

    ch := make(chan int)

    url := "http://some-dummy-url/ping"

    totalHits, _ := strconv.Atoi("50000")

    waitGroup.Add(totalHits)

    for i := 0; i < totalHits; i++ {

        go MakeRequests(url, ch, &waitGroup)

    }

    for i := 0; i < totalHits; i++ {

        if <-ch == 200 {

            count++

        }

    }

    waitGroup.Wait()

    fmt.Printf("Total number of successfull request :" + strconv.Itoa(count))

}

错误

Get http://some-dummy-url/ping: dial tcp 10.120.0.45:80: connect: can't assign requested address

panic: runtime error: invalid memory address or nil pointer dereference

[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x123da32]

goroutine 13425 [running]:

main.MakeRequests(0x7ffeefbff868, 0x22, 0xc0000880c0, 0xc0000ae290)

        /Users/kshitij/go/src/github.com/pingService/healthCheck.go:19 +0xe2

created by main.main

        /Users/kshitij/go/src/github.com/pingService/healthCheck.go:33 +0x136

exit status 2

Answer 1

你必须resp.Body靠近MakeRequests。导致错误的另一个问题是您由于错误而尝试访问 StatusCodewhen is nil。resp这是固定的代码。func MakeRequests(url string, ch chan<- int, wg *sync.WaitGroup) {&nbsp; &nbsp; time.Sleep(time.Duration(rand.Intn(10)) * time.Millisecond)&nbsp; &nbsp; resp, err := http.Get(url)&nbsp; &nbsp; if resp != nil {&nbsp; &nbsp; &nbsp; &nbsp; ch <- resp.StatusCode&nbsp; &nbsp; &nbsp; &nbsp; defer func() {&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; _, err = io.Copy(ioutil.Discard, resp.Body)&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; resp.Body.Close()&nbsp; &nbsp; &nbsp; &nbsp; }()&nbsp; &nbsp; } else {&nbsp; &nbsp; &nbsp; &nbsp; ch <- -1 // because main expect exactly totalHit values in ch&nbsp; &nbsp; }&nbsp; &nbsp; if err != nil {&nbsp; &nbsp; &nbsp; &nbsp; fmt.Println(err)&nbsp; &nbsp; }&nbsp; &nbsp; wg.Done()}

Answer 2

您可以使用以下库：Requests：一个 Go 库，用于减少发出 HTTP 请求时的麻烦（20k/s 请求）https://github.com/alessiosavi/Requests这个想法是分配一个请求列表，然后使用可配置的“并行”因子发送它们，该因子允许一次仅运行“N”个请求。// This array will contains the list of requestvar reqs []requests.Request// N is the number of request to run in parallel, in order to avoid "TO MANY OPEN FILES. N have to be lower than ulimit threshold"var N int = 12// Create the list of requestfor i := 0; i < 1000; i++ {&nbsp; &nbsp; // In this case, we init 1000 request with same URL,METHOD,BODY,HEADERS&nbsp;&nbsp; &nbsp; req, err := requests.InitRequest("https://127.0.0.1:5000", "GET", nil, nil, true)&nbsp;&nbsp; &nbsp; if err != nil {&nbsp; &nbsp; &nbsp; &nbsp; // Request is not compliant, and will not be add to the list&nbsp; &nbsp; &nbsp; &nbsp; log.Println("Skipping request [", i, "]. Error: ", err)&nbsp; &nbsp; } else {&nbsp; &nbsp; &nbsp; &nbsp; // If no error occurs, we can append the request created to the list of request that we need to send&nbsp; &nbsp; &nbsp; &nbsp; reqs = append(reqs, *req)&nbsp; &nbsp; }}此时，我们有一个包含必须发送的请求的列表。让我们并行发送它们吧！// This array will contains the response from the givens requestvar response []datastructure.Response// send the request using N request to send in parallelresponse = requests.ParallelRequest(reqs, N)// Print the responsefor i := range response {&nbsp; &nbsp; // Dump is a method that print every information related to the response&nbsp; &nbsp; log.Println("Request [", i, "] -> ", response[i].Dump())&nbsp; &nbsp; // Or use the data present in the response&nbsp; &nbsp; log.Println("Headers: ", response[i].Headers)&nbsp; &nbsp; log.Println("Status code: ", response[i].StatusCode)&nbsp; &nbsp; log.Println("Time elapsed: ", response[i].Time)&nbsp; &nbsp; log.Println("Error: ", response[i].Error)&nbsp; &nbsp; log.Println("Body: ", string(response[i].Body))}您可以在存储库的示例文件夹中找到示例用法。