RE: [tkined] SNMP to SQL (was: High volume SNMP poller)

From: Szokoli Gabor (szocske@vaskutya.sch.bme.hu)
Date: Mon Feb 21 2000 - 17:55:32 MET


On Thu, 17 Feb 2000, Keith Plourd wrote:

> cc: me, if you don't mind. Thanks.
Me, too.
Or just keep it on the list, before everyone starts "me three" -ing...
The listowner sholud have the final word in that, of course :-)

On the topic:
I beleive that collection-time data processing is the way to scale.
The database is simply loaded with digested data: delta production with
counter overflow and reboot in mind, and all these little pesky tasks
are handled once, when the data is collected.
This way I can set up several "probe" like pollers to feed a single
database, which is separately polled to do reports.
Distributed poller can reduce network inpact, roundtrip time, and UDP loss
sensibilty. (Database inserts are TCP, and hopefuly more compact)
This architecture gives fault tolerance, a poller may die or get restarted
at times w/o any influence on the whole system, I loose a sample, that's
it.

I also plan to facilitate these pollers as sophysticated
"threshold-monitoring" agents generating traps under complex conditions:
In fact, I decided to choose an interpreter partly to be able to put such
code into config. files :-)
That means that not all polled variables may need to be recorded,
archived.

On RDMBS being overkill: I for one need it's features to do reports,
complex queries, and, again, it has to scale well: I can experiment
with something cute and fuzzy like mySQL, and replace it even with a
heavy-hitter like a dedicated Oracle server as the need arises.
SQL is standard and cross-platform enough to do that.
I still need to look at the ODBC tcl extension: any experience earned to
share? :-)

ABout db record format: I need it to be flexibly accessible,
and it does not accumulate infinitly: it gets "condensed" into weekly,
monthly, yearly reports, so I do not beleive the density of the
representation to be an issue.

MRTG seems to be the comparison baseline: My primary goal is to exceed it
in putting the bourden of graphics to a javascript-capable browser. :-)

Szocske

--
!! This message is brought to you via the `tkined & scotty' mailing list.
!! Please do not reply to this message to unsubscribe. To subscribe or
!! unsubscribe, send a mail message to <tkined-request@ibr.cs.tu-bs.de>.
!! See http://wwwsnmp.cs.utwente.nl/~schoenw/scotty/ for more information.



This archive was generated by hypermail 2b29 : Mon Jan 08 2001 - 15:27:36 MET