Child pages
  • Aggregator
Skip to end of metadata
Go to start of metadata

Aggregator

The aggregator will collect information about who is logged in where from two locations:

Linux - poll computers for name of logged in user

Windows - scrape a file that Noyes will give us

The information that has been collected wiil then be sent to Flamongo for storage in the database.

API Communication

The Aggregator will

  • Tell the backend who is currently logged in to a Linux or Windows machine, and provide a timestamp.
  • Poll, with some frequency, each of the lab machines in 102

Listing all machines

On CCIS machines, there is a Perl script at /ccis/bin/hostbase. Hostbase manages all the machines at CCIS.

Linux

We only need to match the 10 minute interval set by the current windows data.

The work flow.

This is the current envisioned workflow.

  1. Fetch all /var/log/wtmp files from the hosts we care about (i.e. lab machines).
  2. Parse them into JSON (via wtmp2json, a small C program in the aggregator repo).
  3. Store the data into the backend.

Duplicate data

We need to be careful not to store duplicate data.

Steps 1 and 2 can be accomplished using a simple shell script (i.e. a for loop of scp then wtmp2json <file>). However, we need this to run continuously.

The data semantics

/var/log/wtmp has different types of records. The ones we will care about are USER_PROCESS, DEAD_PROCESS, and BOOT_TIME. The device parameter will tell us whether we have a remote or a local user ("tty[0-9]" indicates local user, "pts/[0-9]+" indicates a remote user).

We want an aggregate count of local users, remote users, and overall every 10 minutes.

See also

  • The last command uses /var/log/wtmp.
  • The man page for wtmp.

Windows

New format

Upon deliberation with systems, we are now getting login, logout events and not just the sampling points. the new format will be

HOSTNAME:(login|logout):(CCIS-WINDOWS|NUNET):<epoch>

We are responsible for deleting the generated files /proj/crew/labmetrics/....

Old format

Chris Noyes will provide us access to a file see logins to Windows machines in the following format:

Oct 28 16:20:03 BULBASAUR,CCIS-WINDOWS CHARMANDER,CCIS-WINDOWS CHARMELEON,CCIS-WINDOWS NIDOKING,CCIS-WINDOWS NIDOQUEEN,CCIS-WINDOWS NIDORINO,CCIS-WINDOWS PIDGEOTTO,CCIS-WINDOWS PIDGEY,CCIS-WINDOWS PIKACHU,NUNET SANDSLASH,CCIS-WINDOWS VENUSAUR,CCIS-WINDOWS WEEDLE,CCIS-WINDOWS
Oct 28 16:30:04 BULBASAUR,CCIS-WINDOWS CHARMANDER,CCIS-WINDOWS CHARMELEON,CCIS-WINDOWS NIDOQUEEN,CCIS-WINDOWS NIDORINO,CCIS-WINDOWS PIDGEY,CCIS-WINDOWS SANDSLASH,CCIS-WINDOWS VENUSAUR,CCIS-WINDOWS WEEDLE,CCIS-WINDOWS
Oct 28 16:40:03 BULBASAUR,CCIS-WINDOWS CHARMANDER,CCIS-WINDOWS CHARMELEON,CCIS-WINDOWS CLEFAIRY,CCIS-WINDOWS NIDOQUEEN,CCIS-WINDOWS NIDORINO,CCIS-WINDOWS PIDGEY,CCIS-WINDOWS SANDSLASH,CCIS-WINDOWS VENUSAUR,CCIS-WINDOWS WEEDLE,CCIS-WINDOWS

etc.

We will poll this file with some frequency and make API calls to the backend to insert each row into the database.

  • No labels