I am using Node.Js v0.10.2 and process about 500 Links in a loop. The http requests get stuck and i can see a drop of bandwith all the time.
On my 100mbit dedicated server, i only have ~4500KB/s throughput. On my home computer with 64 mbit connection only 1500KB/s throughput.I used slurm for throughput measuring.
I am running a debian 6 with i7 2600 on 16gb ram.
I also tried different configurations of agents and tried out Connection: Close / Keep-Alive, but there still was no improvement.
I really have no clue about where it is lacking...
node server.js
/etc/sysctl.conf:
net.core.rmem_max = 33554432
net.core.wmem_max = 33554432
net.ipv4.tcp_rmem = 4096 16384 33554432
net.ipv4.tcp_wmem = 4096 16384 33554432
net.ipv4.tcp_mem = 786432 1048576 26777216
net.ipv4.tcp_max_tw_buckets = 360000
net.core.netdev_max_backlog = 2500
vm.min_free_kbytes = 65536
vm.swappiness = 0
net.ipv4.ip_local_port_range = 1024 65535
net.ipv4.tcp_fin_timeout = 1
net.ipv4.tcp_tw_recycle = 1
net.ipv4.tcp_tw_reuse = 1
net.core.somaxconn = 35000
net.core.netdev_max_backlog = 32000
net.ipv4.tcp_max_syn_backlog = 64000
fs.file-max = 1000000
ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 127906
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 99999
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 127906
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited