aary@xn--e1aghfa.xn--e1aqbccjfc.xn--p1acf to Technology@lemmy.ml · 2 months agoI want to find users of Heritrix3 enginemessage-squaremessage-square1linkfedilinkarrow-up14arrow-down10file-text
arrow-up14arrow-down1message-squareI want to find users of Heritrix3 engineaary@xn--e1aghfa.xn--e1aqbccjfc.xn--p1acf to Technology@lemmy.ml · 2 months agomessage-square1linkfedilinkfile-text
I have such setup, but with some problems: it always does full crawl (doesn’t do deduplication) I am unable to control crawl order (don’t have enough knowledge)
minus-squareВежливый_Лис@xn--e1aghfa.xn--e1aqbccjfc.xn--p1acflinkfedilinkarrow-up2·2 months agokreuzcrawl — a high-performance Rust web crawling engine.
kreuzcrawl — a high-performance Rust web crawling engine.