The purpose of this page is to compare the results we found from testing both the site-to-site VPN and the software VPN.  To do this, we connected to both VPNs at random times over the course of a week and performed X window forwarding of the Eclipse IDE and a browser window.  We recorded the response time it took for multiple tasks to complete including a button click, text entry, window resize, and highlighting a table row.

Site To Site VPN

• How many times tested
  • Monday
    • 3:00 PM findings
      • lbGUI(Scott)
        resizing limited probably works as below
        2 to ten second delay in window resolution
        1 second delay in response to button press
        3 to 4 second delay to highlight table row
        No delay in text entry.   
    • 4:50 PM findings
      • lbGUI(Scott)
        resizing solved
        2 to ten second delay in window resolution
        1 second delay in response to button press, radio button activation
        3 to 4 second delay to highlight table row
        Maybe a bit slower response on the text edit but admittedly usable 
  • Tuesday
    • 8:50 AM findings
      • Eclipse(Scott) 
        Startup time about a minute and a half to open the application
        Windowing about 2 to 5 seconds.
        Starting a new Java Project
        2 to 3 second lag on right click to package explorer
        10 seconds plus opening java project window
        5 second lag entering project name in the text box, not very usable
        1 to 2 second lag on opening project
        Almost a 20 second lag time opening new class wizard
        Similar lag time in text box for class creation of 5 seconds
        Crashed on class creation
        This version of eclipse is apparently unusable on this kernel version.  It keeps crashing.  I’ll download another version
  • Wednesday
    
    • 2:30 PM findings
      • Eclipse Neon With Java8 (Scott)
        1.5 minute startup, but launched differently
        5 second delay on right click
        3 second wizard startup
        text box lag seems to be absent
        class editing is still jittery.  Irritating, but more usable than delays by seconds we’ve seen. 
        Eclipse runs without crashing — so far. 
  • Thursday
    • 10:30 AM findings
      • Browser (Cory)
      • Opened Firefox through X forwarding on DEV (128.231.245.238)
        Scrolling and button clicking in the browser was about 1 second lag time
        Resizing the screen took about 2, 3, 5 and 10 seconds when trying multiple times
        Typing text was 1/2 delay 
    • 1PM findings
      • Eclipse Neon With Java8 (Scott)
        Startup about 1:30 minutes
        Right click response about 5 seconds
        Wizard startup about 4 - 6 seconds
        1-2 second delays in opening project folders
        check boxes have one second delay
        Text box entries and Text Editing is jittery but no long delays 
  • Friday
    • 4:20PM findings
      • Eclipse Neon With Java8 (Scott)
        Startup about a minute 10 seconds
        Right click response time about 4 seconds: This is the one constant usability issue worth complaining about.
        Window response about 5 seconds
        Text box entry lag is sub second
        Three seconds response to open folder

        Check box response is sub second but noticeable
        Usual jitteriness in the text editor 
• Findings
  • How many times un-usable
    • If it had not crashed on class create, Eclipse Juno would still have had usability issues when with an unacceptable time delay on one occasion.  
    • Other factors add to overall usability including consistently slow wizard window start up and long delays in right click menus.  None of these are quite as bad as text entry lag.
    • Jittery text entry is irritating in the java editor, but it's still usable
  • Eclipse neon did not crash and never showed the unacceptable text entry time delay of Eclipse Juno.  Hard to tell if that was application or network related.
  • Overall S2S is slightly, but noticeably, slower than the software VPN

Software VPN

• How many times tested
  • Monday
    • 3:00 PM findings
      • lbGUI(Scott)
        Resizing works from resize button
        Might be slightly faster window resolution
        button response seems slightly faster
        Notable difference in slower text entry delay 
    • 4:50 PM findings
      • lbGUI(Scott)
        Resizing works from resize button in upper left hand corner.   Cannot manually resize, but it’s not a usability issue show stopper
        2 to ten second delay in window resolution
        button response about the same as above
        Text edit same as above.  
        Not really seeing a difference in lag time this time of day 
  • Tuesday
    • 8:50 AM findings
      • Eclipse(Scott)

        Startup time about 30 seconds faster

        Window startup for new Java Project about 12 secounds

        No Text Entry Lag for project name

        Window startup for new Class project about 5 seconds

        No Text Entry Lag for class name

        Eclipse did not crash on Class creation

        Did crash on class editing

        Will attempt a new eclipse install for the next cycle of tests.  
    • 10:30 AM findings
      • Browser (Cory)
      • Opened Firefox through X forwarding on DEV (ncias-d1224)
        Scrolling and button clicking in the browser was 3-4 seconds lag time
        Resizing the screen took over 10 seconds

• • Wednesday
    • 8:50 AM findings
      • Eclipse Neon With Java8 (Scott)
        half minute startup
        1 to five second window startup
        noticeable but usable text box lag in Project and Class wizards.
        class editing is a little jittery.
        Eclipse running smoothly against jdk 1.8
        jittery editing does not get faster over time.
    • 9:30 AM findings

      • Browser (Cory)
      • Opened Firefox through X forwarding on DEV (ncias-d1224)
        Scrolling and button clicking in the browser was about 1 second lag time
        Resizing the screen took over 10 seconds
  • Thursday
    • 8:30 AM findings
      • Opened Firefox through X forwarding on DEV (ncias-d1224)
        Scrolling and button clicking in the browser was about 2 second lag time
        Resizing the screen took about 3  seconds
        Typing text was 1/2 delay
    • 1PM findings
      • Eclipse Neon With Java8 (Scott)
        50 second startup
        slightly faster right click response
        Wizard response 4 seconds
        project and other folders open sub second
        check boxes have one second delay but are sometimes faster here
        editing is still jittery and about the same as S2S 
  • Friday
    • 11:30 AM findings
      • Browser (Cory)
        Opened Firefox through X forwarding on DEV (ncias-d1224)
        Scrolling and button clicking in the browser was about 2 second lag time
        Resizing the screen took about 2 seconds
        Typing text was < 1/2 delay 
    • 4:20 PM findings
      • Eclipse Neon With Java8 (Scott)
        Startup about a minute 10 seconds
        Right click response time about 5 seconds
        Window response about 5 seconds
        Text box entry is flawless 
        Three seconds response to open folder
        Check box response is sub second but noticeable
        Usual jitteriness in the text editor 
• Findings
  • How many times un-usable
    • Eclipse juno crashed consistently. This had more to do with application fault than network issues.   
    • Other factors add to overall usability including consistently slow wizard window start up and long delays in right click menus.
    • Jittery text entry is irritating in the java editor, but it's still usable
  • Eclipse neon did not crash. 
  • Overall slightly faster than S2S by a second or two in most functions.  Much faster startup time for some reason. 

Conclusion

Site to Site is slower, more prone to unacceptable time delays and much less flexible than software VPN.