amazon web services - Kinesis ProvisionedThroughputExceededException even after sufficient shards -


we have facing provisionedthroughputexceededexception issue while writing data on kinesis stream.

case 1: used single m4.4xlarge (16 core, 64gb mem) instance write data on stream pass 3k request jmeter, ec2 instance provides 1100 request per second, choose 2 shard stream(i.e. 2000 eps). in result able write data on stream without loss.

case 2: further testing had created 10 ec2 m4.4xlarge (16 core, 64gb mem) cluster , 11 shard stream (based on simple calculation 1000eps 1 shard, 10 shard + 1 provision). when test ec2 cluster different request cases jmeter 3, 10, 30 millions. receive provisionedthroughputexceededexception error on our log file.

on jmeter side ec2 cluster provides 7500eps , believe 7500eps stream having 11000eps capacity should not return such error.

could me understand reason behind issue.

check producer side, sure inserting data different shards? "partitionkey" value in putrecordrequest call may you.


Comments

Popular posts from this blog

javascript - How to get current YouTube IDs via iMacros? -

c# - Maintaining a program folder in program files out of date? -

emulation - Android map show my location didn't work -