Check this post (and similar ones in the past):
Market Bots
From our end, it is less ‘detecting them’ and more ‘guessing from a pattern of behaviour’. A busy human updating his orders to sell as fast as he can update quite a lot of orders, and seem bottish. But if you start making a list of ‘order of size X listed Y for time period Z’, and notice that it gets updated every 5 minutes on the dot, for hours, and that there are 50 other orders doing the same … well, if it walks like a bot, talks like a bot, and quacks like a bot…