Tuesday, November 15, 2011

debug CodeDom generated code

I wanted to be able to break into generated code.  The following settings create the pdb and the *.vb files needed.

Dim cpd As New VBCodeProvider()
Dim cp As New Compiler.CompilerParameters
cp.ReferencedAssemblies.Add(
"System.dll")
cp.ReferencedAssemblies.Add(
"System.Drawing.dll")
cp.ReferencedAssemblies.Add(
"System.Windows.Forms.dll")
...etc...
cp.GenerateExecutable = False

cp.IncludeDebugInformation = True
cp.CompilerOptions = "/debug:pdbonly"
cp.TempFiles.KeepFiles = True

The last three line made the difference (Tested with vs2005 debugger.)

agile manifesto

It was probably in Dr. Dobb's Journal that I first became aware of the "Agile Manifesto" and I found the ideas very appealing and natural.  However, ten years later, still at step zero with our development team "getting ready to take classes" -- so we'll see how it goes.  At this point, I'm wondering how well Agile has panned out, and is it appropriate for our project.

Agile Manifesto reads, in its entirety, as follows:

We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:

Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan

That is, while there is value in the items on the right, we value the items on the left more.

So how does eschewing processes fit with our CMMI endeavors?
"following a plan" also very ingrained in government contracting....often "comprehensive documentation", as well.

From the Agile software development class ("Agile Programming and Testing"):

Simple Design:  only doing what is required now.  How do you avoid being trapped in a local minimum, evolutionary dead-end, or painted into a corner?  If you are, how do you get out?  Mixed with pair-programming, is there an architect?  How do you get from A to C if A is refactored into X such that A-B-C is no longer viable or X has relaxed constraints that made C possible.

Do tests get retired?  If a test fails, can you remove it?  Too easy/simple to test: Will you have a lot of no-value-added-tests to maintain?

Managing ("Managing Agile software Development")

Since any method would probably work with the perfect team, how perfect of a team is necessary for agile to work?  (Is it only suitable for a small team of experts?)
Is there a threshold below which an imperfect a team is hurt more by agile?

Our project: the PPT links to an XLS that links to a CACI document about suitability.

VS2010: no experts now?

Physical co-location required?

msbuild 4.0 without VS2010

Probably just trivia, but I was trying to run the new msbuild script on a computer with VS2010 or 4.0 SDK (however it did have 4.0 framework installed).

vbc comes back with an error it can't find ResGen.exe and a very verbose hint as to why involving registry settings and ToolPath setting among other things...none of which turn out to be of any help.

It turns out the solution was to set SDK40ToolsPath="existing 2.0 tools path"

e.g.:

C:\Projects\OAMSIMULATOR>C:\WINDOWS\Microsoft.NET\Framework\v4.0.30319\msbuild proj.vbproj /p:SDK40ToolsPath="C:\PROGRA~1\MID05A~1\SDK\v2.0\bin"

Friday, November 11, 2011

Fast and Easy Enum to text

80x faster

I've been thinking about this issue with text associated with enums since way back in 2003, but I never got around to writing and testing any code to make it easier.  Since I also have a gazillion (and counting) enums in DART, I thought I would finally try to do something about it.  There are about four different methods used in our code to convert from an enum to some kind of text to be displayed:

  1. Enum.ToString, when the desired text is a valid VB identifier (simple, but slow)
  2. An associated array of strings indexed by the enum, fast when the enum can be used to index but extra work to make an array of strings and keep it in sync, etc.
  3. A select case type of statement where the enum is tested and a string is returned; again, extra work.
  4. Encoded enum names (e.g., AlertResponseType.INVEST_2F_7CINTEROG) where valid identifiers are created and then the ToString name is converted to readable text.

I had two ideas on how to do this: 1. Create the array (lookup table) automatically and 2. add readable text to the enum value as an attribute.  He is an example:

Private Enum sample4 'verbose
    <Description("Not Declared")> _
    none
    <Description(
"The First of Two")> _
    One
    <Description(
"The Second of Two")> _
    Two
End Enum
or (I added the descriptions for the speedup text)
Public Enum AlertResponseType As Integer 'encoded
    <Description("OK")>             OK = -1
    <Description(
"ABORT|SHUTDWN")>  ABORT_7CSHUTDWN = 0
    <Description(
"ACCEPT")>         ACCEPT
    <Description(
"ACCEPT|CONTROL")> ACCEPT_7CCONTROL
    <Description(
"ACCEPT|ORDER")>   ACCEPT_7CORDER

The EnumText class below will automatically create an array of text associated with the enum values the first time it is used -- and then just use the existing array thereafter.

Speed is not always the most important thing, but I also did some simple loop tests to see how much faster this technique is:

Enum kind Speedup (x faster)
simple 54.2x
sparse 28.0x
flags 17.7x
encoded 82.6x
   

I've tried the lookup using array search versus dictionaries, dictionaries win out if you have long enums so it probably depends on your project.  There are other considerations that can complicate this if you allow things like Enum.ToString for a byte or integer that isn't defined.  (You can google several solutions to this enum issue, for general and specific cases -- I didn't see our encoded identifier, though)

The speedup is pretty impressive (2000 to 9000 percent), although I actually expected EnumText v. IdentifierToString to be in the 1000Xs.

Here's the code:
You use it like this:
EnumText(Of verbose4).ToText(verbose4.Two))
 EnumText(Of AlertResponseType).ToText(AlertResponseType.ACCEPT_7CCONTROL))
 

It basically does a quick assessment of what kind of enum it is and then creates a converter for it.

Public Class EnumText(Of T)

   Shared Sub New()
     
Dim t As Type = GetType(T)
     
If Not t.IsEnum Then
         Throw New ArgumentException("This is only for enums!")
     
End If
      Dim needsLookup As Boolean 'i.e. a dictionary
      Dim names As String() = System.Enum.GetNames(t)
     
Dim valuesT As T() = DirectCast(System.Enum.GetValues(t), T()) 'for byte, etc.
      Dim values(valuesT.Length - 1) As Integer 
      For n As Integer = 0 To names.GetUpperBound(0)
        
Dim name As String = names(n)
        
Dim fi As Reflection.FieldInfo = t.GetField(name)
        
Dim attrs As System.ComponentModel.DescriptionAttribute() = _
           
DirectCast(fi.GetCustomAttributes(GetType(System.ComponentModel.DescriptionAttribute), False), _
            System.ComponentModel.DescriptionAttribute())
        
If attrs.Length > 0 Then
            names(n) = attrs(0).Description
        
End If
         values(n) = System.Convert.ToInt32(valuesT(n))
        
If Not needsLookup _
           
AndAlso values(n) <> n Then
            needsLookup = True
         End If
      Next
      If t.GetCustomAttributes(GetType(FlagsAttribute), False).Length > 0 Then 'is flags
         'also assumes needsLookup
         _converter = New FlagsConverter(names, values)
     
ElseIf needsLookup Then
         _converter = New LookupConverter(names, values)
     
Else
         _converter = New Converter(names)
     
End If
   End Sub

   Private Class Converter '(Of V)
      Protected names As String()
     
Public Sub New(ByVal names As String())
        
Me.names = names
     
End Sub
      Public Overridable Function ToText(ByVal value As Integer) As String
         Return names(value)
     
End Function
   End Class

   Private Class LookupConverter : Inherits Converter
      Private table As New Dictionary(Of Integer, String)
     
Protected values As Integer()
     
Public Sub New(ByVal names As String(), ByVal values As Integer())
        
MyBase.new(names)
        
Me.values = values
        
''overkill: table? what about just iterate through the array?
         ''enum lists are usually short, right?
         For n As Integer = 0 To names.GetUpperBound(0)
            table(values(n)) = names(n)
        
Next
      End Sub
      Public Overrides Function ToText(ByVal value As Integer) As String
         Dim text As String = Nothing
         If table.TryGetValue(value, text) Then
            Return text
        
Else
            Return value.ToString 'fallback
         End If
      End Function
   End Class

   Private Class FlagsConverter : Inherits LookupConverter
     
Public Sub New(ByVal names As String(), ByVal values As Integer())
        
MyBase.new(names, values)
     
End Sub
      Public Overrides Function ToText(ByVal value As Integer) As String
         If value = 0 Then
            Return MyBase.ToText(0)
        
End If
         Dim sb As New System.Text.StringBuilder
        
Dim index As Integer = Me.values.GetUpperBound(0)
        
Do While index > 0
           
If (value And values(index)) = values(index) Then
               value -= values(index)
              
If sb.Length > 0 Then
                  sb.Append(","c)
              
End If
               sb.Append(names(index))
           
End If
            index -= 1
        
Loop
         Return sb.ToString
     
End Function
   End Class

   Private Shared _converter As Converter
  
Public Shared Function ToText(ByVal value As Integer) As String
      Return _converter.ToText(value)
  
End Function

TESTING

   Private Enum simple1 'simple
      none
      One
      Two
  
End Enum
   Private Enum sparse2 'sparse
      none
      One = 1
      Two = 2
      Four = 4
  
End Enum
   <Flags()> _
  
Private Enum flags3 'flags
      none
      One = 1
      Two = 2
      Four = 4
  
End Enum
   Private Enum verbose4 'verbose
      <Description("Not Declared")> _
      none
      <Description(
"The First of Two")> _
      One
      <Description(
"The Second of Two")> _
      Two
  
End Enum
   Public Enum AlertResponseType As Integer 'encoded
      <Description("OK")> OK = -1
      <Description(
"ABORT|SHUTDWN")> ABORT_7CSHUTDWN = 0
      <Description(
"ACCEPT")> ACCEPT
      <Description(
"ACCEPT|CONTROL")> ACCEPT_7CCONTROL
      <Description(
"ACCEPT|ORDER")> ACCEPT_7CORDER
      <Description(
"ACKNOWL|SHUTDWN")> ACKNOWL_7CSHUTDWN
      <Description(
"ASSIGN")> ASSIGN
      <Description(
"BREAK")> BREAK
      <Description(
"BREAK|ENGAGE")> BREAK_7CENGAGE
      <Description(
"CANTCO")> CANTCO
      <Description(
"CDO")> CDO
      <Description(
"CEASE|MISSION")> CEASE_7CMISSION
      <Description(
"CONFIRM|SHUTDOWN")> CONFIRM_7CSHUTDOWN
      <Description(
"COVER")> COVER
      <Description(
"DCORLTE")> DCORLTE
      <Description(
"DEFER")> DEFER
      <Description(
"ENGAGE")> ENGAGE
      <Description(
"HNDOVER|TO")> HNDOVER_7CTO
      <Description(
"IGNORE")> IGNORE
      <Description(
"INTER­|CONSOLE|HNDOVER")> INTER_AD_7CCONSOLE_7CHNDOVER
      <Description(
"INVEST/|INTEROG")> INVEST_2F_7CINTEROG
      <Description(
"KILL")> KILL
      <Description(
"NEVER")> NEVER
      <Description(
"REJECT")> REJECT
      <Description(
"REQ|CONTROL")> REQ_7CCONTROL
      <Description(
"SCRAM")> SCRAM
      <Description(
"SURVIVE")> SURVIVE
      <Description(
"WILCO")> WILCO
      <Description(
"OVERRIDE")> OVERRIDE
  
End Enum


   
Public Shared Sub Test()
   
Dim s3 As flags3 = CType(5, EnumText(Of T).flags3)
    Debug.Print(
"--Samples--")
    Debug.Print(
"simple1.One  =" & EnumText(Of simple1).ToText(simple1.One))
    Debug.Print(
"sparse2.Four  =" & EnumText(Of sparse2).ToText(sparse2.Four))
    Debug.Print(
"flags3.One Or flags3.Four  =" & EnumText(Of flags3).ToText(flags3.One Or flags3.Four)) 'flags
    Debug.Print("verbose4.Two  =" & EnumText(Of verbose4).ToText(verbose4.Two))
    Debug.Print(
"spares(5)  =" & EnumText(Of sparse2).ToText(5)) 'not defined
    Debug.Print("flags(5).ToString  =" & s3.ToString)
    Debug.Print(
"flags3(5).ToText  =" & EnumText(Of flags3).ToText(s3)) 'flag using integer
    Debug.Print("ACCEPT_7CCONTROL  =" & EnumText(Of AlertResponseType).ToText(AlertResponseType.ACCEPT_7CCONTROL))
    Debug.Print(
"--Speedup (simple)--")
   
Dim sw As Stopwatch = Stopwatch.StartNew
   
For i As Integer = 1 To 1000000
      
Dim s As String = simple1.Two.ToString
   
Next
    Dim sw1 As Long = sw.ElapsedTicks
    sw.Reset() : sw.Start()
   
For i As Integer = 1 To 1000000
      
Dim s As String = EnumText(Of simple1).ToText(simple1.Two)
   
Next
    Dim sw2 As Long = sw.ElapsedTicks
    Debug.Print(
"{0} {1} {2:0.0}", sw1, sw2, sw1 / sw2)
    Debug.Print(
"--Speedup (sparse)--")
    sw.Reset() : sw.Start()
   
For i As Integer = 1 To 1000000
      
Dim s As String = sample2.Four.ToString
   
Next
    sw1 = sw.ElapsedTicks
    sw.Reset() : sw.Start()
   
For i As Integer = 1 To 1000000
      
Dim s As String = EnumText(Of sample2).ToText(sample2.Four)
   
Next
    sw2 = sw.ElapsedTicks
    Debug.Print(
"{0} {1} {2:0.0}", sw1, sw2, sw1 / sw2)
    Debug.Print(
"--Speedup (flags)--")
    sw.Reset() : sw.Start()
   
For i As Integer = 1 To 1000000
      
Dim s As String = s3.ToString
   
Next
    sw1 = sw.ElapsedTicks
    sw.Reset() : sw.Start()
   
For i As Integer = 1 To 1000000
      
Dim s As String = EnumText(Of sample3).ToText(s3)
   
Next
    sw2 = sw.ElapsedTicks
    Debug.Print(
"{0} {1} {2:0.0}", sw1, sw2, sw1 / sw2)

    Debug.Print(
"--Speedup (encoded)--")
    sw.Reset() : sw.Start()
   
For i As Integer = 1 To 1000000
      
Dim s As String = IdentifierToString(AlertResponseType.INTER_AD_7CCONSOLE_7CHNDOVER.ToString)
   
Next
    sw1 = sw.ElapsedTicks
    sw.Reset() : sw.Start()
   
For i As Integer = 1 To 1000000
      
Dim s As String = EnumText(Of AlertResponseType).ToText(AlertResponseType.INTER_AD_7CCONSOLE_7CHNDOVER)
   
Next
    sw2 = sw.ElapsedTicks
    Debug.Print(
"{0} {1} {2:0.0}", sw1, sw2, sw1 / sw2)

   End Sub
   Shared Function IdentifierToString(ByVal sIdentifier As String) As String
      'NAME: Devon Terminello
      'DATE: March 2004
      'PURPOSE: This function replaces _[hex digit][hex digit] with the asc character that
      '         corresponds with the 2 hex digits. NOTE: This gives me a way to encode
      '         characters into an identifier, which would not otherwise be visible
      Dim sRet As String = ""
      Dim iIndexId As Integer, iChr As Integer
      If Not sIdentifier Is Nothing Then
         Do While iIndexId < sIdentifier.Length
           
If sIdentifier.Chars(iIndexId) = "_"c Then
               iChr = Integer.Parse(sIdentifier.Substring(iIndexId + 1, 2), Globalization.NumberStyles.HexNumber)
               sRet &= Chr(iChr)
               iIndexId += 3
           
Else
               sRet &= sIdentifier.Chars(iIndexId)
               iIndexId += 1
           
End If
         Loop
      End If
      Return sRet
  
End Function

End
Class

 
It prints out:

--Samples--
simple1.One  =One
sparse2.Four  =Four
flags3.One Or flags3.Four  =Four,One
verbose4.Two  =The Second of Two
spares(5)  =5
flags(5).ToString  =One, Four
flags3(5).ToText  =Four,One  <-- I do my flags backwards compared to MS, but it matches how I assign bits better I think
ACCEPT_7CCONTROL  =ACCEPT|CONTROL

Other things of note:
Negative valued enums, MS uses unsigned sort, so GetNames will have the negative numbered enums at the end.
Since I set and think of flag bits going right to left, that's how I print out my text, too,

Tuesday, November 1, 2011

DirectoryInfo needs an Empty()

I want to have an empty directory to save the results of some re-runnable process (e.g., one that dumps PNGs of all the controls on a form for documentation purposes).

di = New IO.DirectoryInfo("SomeDirectoryToExistAndBeEmpty")
If di.Exists Then
   For Each fi As IO.FileInfo In di.GetFiles
      fi.Delete()
  
Next
   For Each dir As IO.DirectoryInfo In di.GetDirectories
      dir.Delete(
True)
  
Next
Else
   di.Create()
End If

(in this case there are no read-only files or any special attributes since I am generating and regenerating the files myself)

Form_Load and BeginInvoke

I inherited a form that did a lot of processing during the load event and I had a problem with displaying a mostly blank form...

Private Sub Form_Load(ByVal eventSender As System.Object, ByVal eventArgs As System.EventArgs)
       DrawInitialScreen() 'draw something to see
       If DataFileProvided Then ' e.g. via Args, then also do this stuff:
          Me.BeginInvoke(New MethodInvoker(AddressOf Me.Refresh)) 'this will show the screen, then
         
Me.BeginInvoke(New MethodInvoker(AddressOf OpenFile))   'this will do a lot of work
      
End If
    End If
End Sub

Using BeginInvoke allows the load event to finish, then do the Screen refresh showing something besides a lot of white space.
The second BeginInvoke allows the Refresh to finish before starting the long work of processing a data file.

Another approach would be to load the file on a worker thread and sync things up when done, but since the user can't do anything until the file is loaded a simple progress bar was sufficient here.

Another combination that appears to work that I somehow missed* before going to BeginInvoke is this:

Private Sub Form_Load(ByVal eventSender As System.Object, ByVal eventArgs As System.EventArgs)
       DrawInitialScreen() 'draw something to see
       If DataFileProvided Then ' e.g. via Args, then also do this stuff:
          Me.Visible = True
          Me.Refresh
          OpenFile
       End If
    End If
End Sub

*I find it hard to believe that Refresh is not the first thing I tried.  I seem to recall it just wouldn't work with something horrible like a DoEvents.  Which was fatal in this app because the legacy code would start processing mouse-moves trying to use the data that hasn't been loaded yet.  There is also a progress meter that needs to refresh during the load (which works just fine), although there was a textbox that would not -- both on a toolstrip.  I believe the progress uses its parent control to refresh, I'll have to look at the textbox again.

Does Refresh always work?

I recall instances in the past where I couldn't succeed with Refresh, but so far I can't find an example.  I redid the project above that prompted this investigation with Refresh and it worked this time.  I don't see why it shouldn't work, here is Refresh()



Public Overridable Sub Refresh()
Me.Invalidate(True)
Me.Update
End Sub


Update is just a wrapper around UpdateWindow.  I would say Invalidate() puts a WM_PAINT in the queue for each control, then Update() scans the queue and sends the WM_PAINT to the control. From MSDN (makes it sound a little different than what I am saying but I think they are simplifying for clarity):



"The UpdateWindow function updates the client area of the specified window by sending a WM_PAINT message to the window if the window's update region is not empty. The function sends a WM_PAINT message directly to the window procedure of the specified window, bypassing the application queue. If the update region is empty, no message is sent."



Sounds good.  Under what conditions wouldn't this paint the whole form?  Maybe my previous problems were actually bad custom controls or some-other non-Windows issue and this whole entry was wasted on a non-problem.



What's old is new again




However, check this out:




http://geekswithblogs.net/NewThingsILearned/archive/2008/08/25/refresh--update-wpf-controls.aspx 

 




Also, Hide-and-see


When hiding a form and then re-showing it, e.g., Show() or Visible=True; is a Refresh needed there, too? It would seem so.