credentiality: 03/01/2015

Monday, March 23, 2015

Steam Linux audio problems

No audio at all from steam games? Try 'pavucontrol'. It'll show you when games are trying to play audio and where they're playing it to. In the "configuration" tab I had to disable built-in audio and switch my "HDA NVidia" device to "Digital Surround 5.1 (HDMI) Output" from "Digital Stereo (HDMI) Output".

Sunday, March 22, 2015

GQL to CSV exporter for Google App Engine

The App Engine admin panel will let you run GQL queries against your datastore, but it won't let you download the results as a CSV. So I wrote a [very] quick and [very] dirty handler that does, which also has the handy side effect that you can put your own access controls on it. That means you can give someone on your team read-only access to your datastore without also giving them access to the admin panel.

If you're using db instead of ndb, I believe the main thing to change is ndb.gql() to GqlQuery().

 # Very quick and dirty example of how to provide unfettered read  
 # access to your datastore with export to CSV. Be sure to add appropriate  
 # access controls and watch out for security risks (like XSS)  
 #  
 # Don't forget to:  
 # from google.appengine.ext import ndb  
 # from google.appengine.ext.ndb import metadata  
 class GqlPage(webapp2.RequestHandler):  
  def get(self):  
   limited = True  
   row_limit = 1000  
   # Tricky to distinguish absence of 'limit' checkbox when you  
   # first hit the URL from when you submitted with an unchecked box  
   if self.request.get('download', 'nope') != 'nope' and \  
     self.request.get("limit", "nope") == "nope":  
    limited = False  
   
   query = self.request.get('query', "empty")  
   
   is_csv = False  
   if self.request.get('download') == 'Download':  
    is_csv = True  
    self.response.headers['Content-Type'] = "text/csv"  
   else:  
    self.response.write('<html><body>')  
    self.response.write('<form action="/gql" method=POST>')  
    self.response.write('<textarea name=query rows=10 cols=80 placeholder="select * from...">')  
    if query != "empty":  
     self.response.write(cgi.escape(query))  
    self.response.write('</textarea><br>')  
    self.response.write("<input type=submit name=download value=View>")  
    self.response.write('<input id=foo type=submit name=download value=Download')  
    self.response.write(' onclick="document.getElementById(\'results_div\').innerHTML=\'\';">')  
   
    self.response.write("Limit response to " + str(row_limit) + " rows:")  
    self.response.write("<input name=limit type=checkbox ")  
    if limited:  
     self.response.write("checked")  
    self.response.write('><br></form>')  
   
    self.response.write("Examples for available tables:<br>")  
    for kind in metadata.get_kinds():  
     self.response.write("select * from " + kind + "<br>")  
   
    self.response.write('<br><div id="results_div"><pre>')  
   
   if query != 'empty' and query != '':  
    results = []  
    if limited:  
     results = ndb.gql(query).fetch(row_limit)  
    else:  
     results = ndb.gql(query).fetch()  
   
    writer = csv.writer(self.response.out)  
   
    row_count = 0  
    first_row = True  
    for row in results:  
     row_dict = row.to_dict()  
     keys = sorted(row_dict.keys())  
   
     # Write column labels as first row  
     if first_row:  
      first_row = False  
      self.response.write("#")  
      writer.writerow(keys)  
   
     values = []  
     for k in keys:  
      value = str(row_dict[k])  
      if is_csv:  
       values.append(value)  
      else:  
       values.append(cgi.escape(value))  
   
     writer.writerow(values)  
   
     row_count += 1  
     if not is_csv and limited and row_count == row_limit:  
      self.response.write("\n[Truncated at " + str(row_limit) + " lines]")  
   
   if not is_csv:  
    self.response.write('</pre></div>')  
    self.response.write("</body></html>")  
   
  def post(self):  
   return self.get()

Friday, March 20, 2015

Linux data acquisition with an Agilent Infiniium MSO9404A oscilloscope

I need to capture some analog data. My coworker has a National Instruments USB data acquisition module, but as far as I can tell, the drivers are all proprietary and focused on using Labview.

So instead I used a fancy 9000-series Agilent Infiniium scope. I plugged my laptop's ethernet cable into the scope, then used the Windows control panel on the scope to set its IPv4 address to 192.168.1.1. I set my laptop to 192.168.1.2, and found that I could ping the scope. So far so good.

Back in the day, test equipment was controllable over a serial GPIB bus. Today's fancy gear uses the same conventions over TCP/IP. There's a confusing mess of acronyms like VXI-11 and VISA, and a bunch of half-baked libraries and crappy-looking system drivers that appear to be required to use them. pyvisa looks nice, but wants a proprietary National Instruments driver distributed as a .iso(!). Not my cup of tea.

Then I ran across this MATLAB example that's basically just chatting with the device over TCP/IP. That led me to Agilent's Programmer's Reference guide for the 9000-series scopes.

After fighting with the 1100-page manual for a few hours, I came up with the following settings that let me get samples at a specific rate like I would from an ADC.

Note that you can also interactively talk to the scope using nc or telnet to port 5025 while you're experimenting.

 #!/usr/bin/python  
   
 # Using ethernet to talk to an Agilent Infiniium MSO9404A  
 # See also the "Infiniium 9000A Programmer's Reference"  
   
 import socket  
 import sys  
   
 sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)  
 sock.connect(("192.168.1.1", 5025))  
   
 def cmd(s):  
  sock.send(s + "\n")  
   
 def resp():  
  r = sock.recv(1048576)  
  while not r.endswith("\n"):  
   r += sock.recv(1048576)  
  return r  
   
 print "Querying scope"  
   
 cmd("*IDN?")  
 print "Scope identifies as: ", resp(),  
 cmd("*RST")  
   
 # This doesn't seem to affect the samples we receive  
 cmd(":timebase:range 1E-6")  
 cmd(":timebase:delay 0")  
   
 cmd(":channel1:range 5")  
 cmd(":channel1:offset 2.5")  
 cmd(":channel1:input dc")  
   
 cmd(":trigger:mode edge")  
 cmd(":trigger:slope positive")  
 cmd(":trigger:level chan1,2.5")  
   
 cmd(":system:header off")  
   
 cmd(":acquire:mode rtime") # Realtime mode; don't average multiple passes  
 cmd(":acquire:complete 100")  
   
 cmd(":waveform:source channel1")  
 cmd(":waveform:format ascii")  
   
 cmd(":acquire:count 1")  
 cmd(":acquire:srate:auto off")  
 cmd(":acquire:srate 4000")  
 cmd(":acquire:points:auto off")  
 cmd(":acquire:points 16")  
 # This was on by default, and took me a long time to figure out why  
 # I was getting ~16x the number of samples I requested.  
 cmd(":acquire:interpolate off")  
   
 cmd(":digitize channel1")
# This should block until the capture is done, since we used :digitize
 cmd(":waveform:data?")  
   
 sample_string = resp().rstrip().rstrip(",")  
 ascii_samples = sample_string.split(",")  
   
 samples = []  
 for f in ascii_samples:  
  try:  
   samples.append(float(f))  
  except:  
   print "Couldn't convert:", f  
   
 print "Got ", len(samples), " samples. Values 1-10:", samples[:10]

credentiality