Best Log File Analyzers for SEO: Ten Options Compared
Server logs show what Googlebot and other bots actually request, not what a crawler guesses. I compared ten ways to analyse them, from a desktop licence to open-source tools and enterprise platforms, on price, log limits and fit.
- Vendor pages read, Nov 2026
- Free and open-source options separated
- Unreadable pages marked “Not verified”
// disclosure I do not sell any of these tools and no provider can pay for a place or a score. Read the full disclosure.
| tool | price or limit |
|---|---|
| Screaming Frog Log File Analyser | GBP 99/yr; free to 1,000 events |
| GoAccess | Open source, MIT licence |
| JetOctopus Growth | $349/mo, 2M log lines |
| Oncrawl | Pricing not shown |
| Elastic | Self-managed or cloud |
// quick answer For most small and mid-size sites, Screaming Frog’s Log File Analyser is the cheapest dedicated tool. If you are comfortable on the command line, GoAccess is free. Cloud platforms such as JetOctopus cost far more and make sense for large sites. Several enterprise vendors would not show prices, so I mark them Not verified.
How I chose
I read vendor pages and scored price clarity, log limits, formats supported, bot verification and fit. Scores are editorial out of 10; I ran no hands-on tests. Where a page would not load or showed no price, I say so and score the tool about 6.0.
Why analyse logs
Crawlers show your site’s structure; logs show real bot behaviour. That helps with crawl budget on big sites, orphan pages and wasted requests. Google’s crawler documentation says its crawlers identify themselves by user-agent, IP address and reverse DNS hostname, and that logs may show many IP addresses, so check all three before trusting a hit (see Google’s overview and my guide to verifying Googlebot). For a free first look, Search Console’s Crawl Stats report helps, and my crawler comparison covers the crawl side.
Comparison table
| Tool | Price | Type | Log limits | Score |
|---|---|---|---|---|
| Screaming Frog LFA | GBP 99/yr | Desktop | 1,000 events free; unlimited paid | 8.4 |
| GoAccess | Free (open source) | Terminal or HTML | No cap stated | 7.6 |
| JetOctopus | From $349/mo | Cloud | 2M to 25M lines a month | 7.5 |
| Crawl Stats report | Free | Search Console | Google’s crawl only | 7.2 |
| Elastic | Free self-managed; cloud priced | Platform | Your hardware | 7.0 |
| Oncrawl | Not shown | Cloud | Not verified | 6.0 |
| Botify | Not verified | Enterprise | Not verified | 6.0 |
| Lumar | Not verified | Enterprise | Not verified | 6.0 |
| Semrush | Not verified | Suite feature | Not verified | 6.0 |
| Seolyzer | Not verified | Cloud | Not verified | 5.8 |
The ten reviews
Screaming Frog Log File Analyserbest value
Desktop log analyser
- PAID
- GBP 99 per year; unlimited events
- FREE
- 1,000 log events, one project
- FORMATS
- Apache, W3C Extended (IIS, NGINX), AWS ELB
It verifies search engine bots automatically and shows most and least crawled URLs, crawl frequency, errors, redirects and orphan URLs. The page says reports cover AI bots such as ChatGPT but not which are verified. Windows, macOS and Linux are supported; the paid maximum depends on disk space.
- Low price
- Free tier to test
- Desktop only
- Yearly renewal
GoAccessfree
Open-source log viewer
- LICENCE
- MIT; no price stated
- FORMATS
- Apache, Nginx, S3, ELB, CloudFront, Caddy
- OUTPUT
- Terminal, HTML, JSON, CSV
It offers real-time updates in the terminal and a browser dashboard. The page describes a general web log analyser; it does not mention SEO-specific bot verification, so you would filter and verify bots yourself.
- Free
- Real-time
- Not SEO-specific
- Command-line setup
JetOctopuscloud, priced high
Crawler and log analyzer
- GROWTH
- $349/mo; 2M log lines a month
- PRO
- $809/mo; 8M (in-house) or 6M (agency)
- ULTRA
- $1,559/mo; 25M or 12M
The page shows a 25% discount toggle and does not say which billing period the prices apply to. The trial is opened after a demo call, with no card needed. Unlimited is quote-only.
- Cloud dashboards
- Trial without a card
- Expensive for small sites
- Billing period unclear
Crawl Stats reportfree baseline
Google Search Console
- COST
- Free
- SCOPE
- Google’s own crawling only
- SOURCE
- Not a server log
I did not re-read Google’s page for this guide, so treat my description as general. It will not show Bing, AI bots or fake bots, but it is a free sanity check before you invest in logs. See my Crawl Stats guide.
- Free
- No server access
- Google only
- Summarised data
Elastic (ELK)
Search and log platform
- SELF-MANAGED
- License-based by nodes and RAM; free download
- CLOUD
- Serverless and Hosted, usage or resource based
- TRIAL
- Free trial on cloud
The pricing page shows no dollar amounts. It is a general logging stack, so building SEO views takes engineering time.
- Scales
- Flexible
- Needs engineers
- No prices on page
OncrawlNot verified
Crawler with log analysis
- PRICING
- Page lists no plans or prices
- MODEL
- “Different plans” per its page
- LISTINGS
- A third-party list says from $69/mo; unconfirmed
The only vendor statement I could read is that plans are adapted to use cases. I cannot verify limits.
- Combines crawl and logs
- No visible pricing
BotifyNot verified
Enterprise SEO platform
- PAGE
- Product page returned an error
- PRICE
- Not found
- POSITION
- Described as enterprise by third parties
I could not read the vendor page, so I have no verified prices or limits.
- Enterprise focus reported
- Unverified
LumarNot verified
Enterprise technical SEO
- PAGE
- Product page returned an error
- PRICE
- Not found
- FORMERLY
- DeepCrawl, per third parties
No verified pricing or log limits. Third parties describe it as enterprise technical SEO software.
- Enterprise focus reported
- Unverified
Semrush Log File AnalyzerNot verified
Feature inside a suite
- HELP PAGE
- Returned an error
- SUITE PRICE
- SEO plan $139/mo per its pricing page
- FEATURE
- Described by third parties only
I could not confirm requirements, limits or which plans include it. Buying a whole suite for this alone is hard to justify. See my Semrush review.
- Inside a suite
- Unverified details
SeolyzerNot verified
Log and crawl tool
- PAGE
- Not read
- LISTING
- Pricing on request, per a directory
- FOCUS
- Crawl budget on large sites, per that directory
I only saw a third-party directory entry. I cannot verify anything about price or features.
- Reported crawl budget focus
- Unverified
Head to head
Screaming Frog vs GoAccess. The first adds bot verification and SEO views for GBP 99 a year; GoAccess is free but general-purpose.
Screaming Frog vs JetOctopus. $349 a month is far above a GBP 99 yearly licence; the cloud tool adds shared dashboards and scale.
Logs vs Crawl Stats. Crawl Stats shows Google only; logs show all bots including fake ones.
Elastic vs a dedicated tool. Elastic needs building; dedicated tools ship ready reports.
Enterprise vs desktop. Botify, Lumar and Oncrawl hide prices; test the cheaper tools first.
Suite feature vs standalone. Pay for a suite log tool only if you already use the suite.
Get and read your logs
- Ask your host for raw access logs and the retention period.
- Export 30 days if you can; one day is too thin.
- Load them in a tool that supports your format (Apache, W3C, ELB).
- Verify bots by IP and reverse DNS, not user-agent alone.
- Compare most-crawled URLs to your priority pages and fix waste.
Six mistakes to avoid
Trusting user-agent strings
Anyone can fake Googlebot.
Using a tiny sample
A day of logs misleads.
Ignoring CDN logs
Origin logs may miss cached hits.
Overbuying
Small sites rarely need a cloud platform.
Privacy blind spots
Logs hold IP addresses.
No action list
Reports are not fixes.
What this guide does not tell you
It does not measure speed, accuracy or support, and I ran no tests. Oncrawl, Botify, Lumar, Semrush and Seolyzer details could not be verified on vendor pages. The Crawl Stats description is general and not taken from a fresh read of Google’s page.
Key terms
- Log event
- One line recording a request to your server.
- Crawl budget
- How much a bot crawls on your site.
- Reverse DNS
- Looking up a hostname from an IP address.
- Orphan URL
- A page with no internal links.
- W3C Extended
- A log format used by IIS and others.
- AI bot
- A crawler run by an AI company.
Related guides
FAQ
What is the best log file analyzer for SEO?
For most sites, Screaming Frog’s Log File Analyser at GBP 99 a year, with a free 1,000-event version.
Is there a free log analyzer?
GoAccess is open source under the MIT licence, and Screaming Frog’s free version handles 1,000 events.
Do I need server access?
You need raw access logs from your host or CDN.
How do I verify Googlebot?
Check user-agent, IP address and reverse DNS together, per Google’s crawler documentation.
Which formats do tools support?
Screaming Frog lists Apache, W3C Extended and AWS ELB; GoAccess lists many including Nginx and CloudFront.
Are logs worth it for a small site?
Often not at first. Crawl Stats and a crawler cover most small-site needs.
Can logs show AI bots?
Yes, if the tool recognises them; Screaming Frog says its reports cover AI bots.
Are these prices current?
They were read in Nov 2026 and change often.
How this guide is funded
No provider can pay for a place or a score. Scores are editorial opinions from public vendor pages. The site sells SEO services and uses Yoast SEO, so I have a commercial interest in SEO topics. I link only to authority sources such as Google documentation and the FTC’s Endorsement Guides. Vendors are named, not linked.




