So, the "Find" and "Find/Replace" tools aren't an important part of Visual Studio, are they?

I recently installed the RTM of Visual Studio 2008 Professional on my Windows XP x64 machine, leaving Visual Studio 2005 installed, as well, just in case things went awry with the new version. I've been using the early release versions of 2008 for sometime now, and have been impressed with the performance improvements and many of the new features, so I didn't think that I'd have too many problems with or misgivings about the upgrade.

Well, after the install I was working in the IDE and used the trusty Ctrl-F combo to open up the "Find" dialog. Visual Studio 2008 immediately conked on me. Totally conked. I opened it back up and tried Ctrl-F again and it conked again.

At this point I was ready to go back to 2005 (I was pretty disappointed in the much-hyped JavaScript debugging already), but I decided to do a quick Google search first. After performing the search, I ended up on the Visual Studio feedback site at the "Visual Studio 2008 RTM Search Crashes" bug report.

For those of you who aren't into reading bug reports, I'll summarize the contents: If you use more than one monitor and a utility (UltraMon in my case, but could be a utility that comes with your video card or any other third party utility) that helps you manage the multiple monitor setup, Visual Studio 2008 will crash when you open up the "Find" and "Find/Replace" dialog. This bug is apparently not a problem with Visual Studio 2008, but a newly-discovered (or at least newly-publicized) bug in Windows XP x64.

After reading through the multiple posts that were associated with the bug report, it became painfully obvious that I had only two choices: either uninstall my baby UltraMon or go back to using Visual Studio 2005. I didn't even have to think about it; I immediately went back to using 2005 and accepted my fate.

Today I randomly decided to check to see if an update for UltraMon had been released, and lo and behold it had! And talk about lucky - in this newest release (3.01 = beta 2 of 3.0) Realtime Soft decided to take some of the Windows 'hooks' out, meaning that the Windows XP x64 bug that was causing my Visual Studio 2008 Ctrl-F issues was fixed for me immediately!

I've run into this problem several times before, but have always forgotten to document it. Because of this, every time that I get to work setting up a new development environment and then try to compile an ASP.NET 2.0 application that sits on a network share, I have to relearn my lesson. Well, this time I'm going to stick this tidbit of information here, in hopes that I'll remember it next time I go through this, and I hope that it helps someone else out there, as well.

Basically, the issue pops up when you try to compile an ASP.NET 2.0 application that lives on a network share in Visual Studio 2005. Here's the error: "ASP.NET runtime error: Request for the permission of type 'System.Web.AspNetHostingPermission, System, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089' failed."

The AspNetHostingPermission Class protects public types in the System.Web namespaces. If you're developing, you can set the trust level on your development machine and move on. If you need to access these public types during runtime, you'll need to change the trust level in your application's web.config. By default, the AspNetHostingPermission is only assigned to applications that are running under Full trust.

Luckily, the fix is simple (once you find it). On your development machine, open up the Microsoft .NET Framework 2.0 Configuration utility, either by getting to it through the "Administrative Tools" section of Control Panel or through the "Administrative Tools" link on your Start menu. Note that if you don't have "Administrative Tools" installed, you can download it from Microsoft. It's included with the Windows Server 2003 SP1 Admin Pack.

Once into the Microsoft .NET Framework 2.0 Configuration Utility, click the "+" next to "My Computer", click on "Runtime Security Policy", and then click on "Adjust Zone Security" underneath the "Tasks" section in the right-hand pane. In the "Security Adjustment Wizard" dialog, verify that the "Make changes to this computer" radio button is selected and click "Next>" to advance in the dialog. Select "Local Intranet" from the top pane and move the slider on the bottom up to the very top, or "Full Trust". Click "Next>" and then finish to verify and submit the change. If you still have Visual Studio 2005 open, you'll need to close it and reopen it. Recompile, and you should be good.

Thanks to Jason Gaylord's blog entry for pointing me (back) in the right direction.

A quick followup to a post I made a couple of weeks ago about "Improving a Web Application's Page Load Time and Responsiveness": Thanks to Omar Shahine, I just discovered Yahoo!'s "Thirteen Simples Rules for Speeding Up Your Web Site". I linked to the Yahoo! User Interface Blog in my performance blog post, but failed to mention the Yahoo! Developer Network (YDN).

The YDN is another great resource for web developers. It provides tons of helpful tips and real-world examples to help developers build better web applications. The Yahoo! User Interface (YUI) library is probably my favorite resource that the YDN provides.

The YUI is a set of CSS libraries and  elegant JavaScript controls that are made available under the BSD license for all to use. The JavaScript outperforms the controls delivered with Microsoft's ASP.NET AJAX control toolkit (I know, in a way it's kind of like comparing apples and oranges, but I'm talking about quality here), and the documentation and community that support the controls are both organized and helpful. Over the last couple of months, I have made it a practice to include the YUI "core CSS foundation" in all of my web applications, as they make developing standards-compliant and consistent websites *much easier. If you didn't know, all browsers have their own built-in CSS quirks that, if not neutralized, can make web design a very frustrating process. If you include YUI's "core CSS foundation" in your application, these quirks are neutralized from the beginning, meaning that you can truly start developing (building) on level ground.

Another cool YDN resource that I just discovered is the "YSlow" Plugin for Firefox. It integrates directly with my favorite Firefox web development add-on, Firebug, and is an extremely helpful way to judge how your website lives up to the "Thirteen Simple Rules" mentioned above. After using it to analyze this site (nateirwin.net), I know that I definitely need to spend more time optimizing it (as I've known for quite some time), and I'll try to make this a higher priority. The site is scheduled for a complete overhaul here in the near future anyways, so maybe I'll wait until then.

Well, I could sing YDN's praises all night, but I'll save that for a dedicated post later on.

Brief Introduction

Microsoft's ADO.NET team recently released a new project, as a Community Technology Preview (CTP), through Microsoft's Live Labs. The project, codenamed "Astoria", is a technology that allows developers to easily build and deploy data services (either on their own servers or to Microsoft's hosted "Astoria Online Service") that are accessible to applications through HTTP requests. This idea is not necessarily new - there are other ways to build a web services interface - but the ability to easily build a data service for SQL Server that is accessible through HTTP is definitely groundbreaking.

As the "Astoria" site preaches: this type of service is especially useful for AJAX-based applications that work hard on improving the user experience, through the use of Flash or Silverlight. Once an "Astoria" data service is published, you can query the underlying data structure by passing parameters in the URI. For example (and this is taken from the "Astoria" Overview document that is available on the Live Labs website):

To access all customers in a database:

http://myserver/data.svc/Customers

To access a customer whose key is "ALFKI":

http://myserver/data.svc/Customers[ALFKI]

To find all orders that are associated with a customer whose key is "ALFKI":

http://myserver/data.svc/Customers[ALFKI]/Orders

And so on, so forth. You can get much more focused than the above examples. You can drilldown into related data, filter out records, sort records, and even page records.  You get the picture, right? It really is a powerful way to interact with a database.

How does "Astoria" make it so simple? Well, when you build an "Astoria" data service with Visual Studio (and, by the way, you need Visual Studio "Orcas"/2008 beta 1 to install the "Astoria" framework and build these types of data services) you build an Entity Data Model (EDM). "Astoria" then uses this model to wade through the underlying database and figure out how data are related. It can then use this simplified information to serve the data and make them accessible through the RESTful interface. "Astoria" can currently represent data in XML, JSON, and (as Pablo Castro, Microsoft's ADO.NET technical lead, mentions in a really great presentation that he gave at MIX 2007) *very experimentally in RDF+XML. XML support is probably the most accepting, but JSON is, of course, perfect for use with AJAX applications.

Helpful/Interesting Resources

I find "Astoria", and the underlying concept of taking complex data and making it broadly accessible intriguing. Again, I understand that this is not new territory. SOAP and other web services are a more powerful way of accessing data, but I see "Astoria"'s restful interface meeting the needs of many. The ability to easily publish a database and make it accessible to the web is something that will allow many developers to build richer (and lighter-weight) web (and mapping!) applications.

• Project "Astoria" Homepage
• Pablo Castro's "Astoria" Presentation at Mix 2007
• John Udell's "A Conversation with Pablo Castro about Astoria's RESTful Data Services"

I work with DataSets a lot in ASP.NET, so I was very surprised when I added a new query to an existing TableAdapter and got an error "Failed to enable constraints. One or more rows contain values violating non-null, unique, or foreign-key constraints" from the new query. I knew that the query was fine, as I executed it with set parameters and received the expected results. I even tried setting EnforceConstraints to false on the dataset, but still no luck.

Then I found a remarkably helpful blog post on this very same error. Well, to clarify: the blog post (which was really more a plea for help than anything else) didn't help me much, but the comments had a ton of suggested solutions. I slowly picked my way through them, one by one, working through some and skipping over others. I finally found a solution to my problem at the bottom of the page.

The schema that was returned from the suspicious query didn't match the schema returned by the TableAdapter. The solution was to create a new TableAdapter that returned the same schema as my query. After that, everything worked as expected.

The thing is, I never received an error message when adding the new query to the TableAdapter, and the behavior doesn't seem consistent at all, as I have several other TableAdapters that have multiple queries which return different schemas. So, I'm still looking for a *real solution (if it exists). This is one of those frustrating problems that I'd really like to get my arms around.

Anybody else dealt with this?

UPDATE: You can now download this blog post.

My apologies for the super-long post; I was originally going to break this up into multiple posts, but my week at the conference filled up very quickly and I couldn't find time to write out all my thoughts and experiences and post them daily. To try and facilitate reading of this post, I have, however, broken it up by day and by technical workshop. And at the end of the post I have written about some miscellaneous tidbits that I discovered over the course of the week. I expect that in the coming weeks I'll be writing more in-depth about many of the topics that I am able to cover only briefly below, as many of the developments have major ramifications for anyone who works with ESRI software.

Judging by a recent post from an austere colleague, I guess that being a participant at the ESRI User Conference (from here on referred to as the UC) puts me (and many others - you know who you are) in a position that brings an inherent level of responsibility. I'll be honest, though, I wasn't aware of these overarching responsibilities when I signed up; my apologies for not reporting the juicy details sooner. Fortunately, though, my colleagues seem to have adequately filled the void that I unknowingly contributed to over this past week.

That said, I walk out of San Diego a week older and with a number of news items, tidbits, and rumors - enough, at least, to get me pretty excited. I'm not sure that there were any monumental announcements or "WOW!" moments this year (a lot of the big announcements were spoiled by Jack's answers to the Questions and Answers that were released the week before the UC), but ESRI and some of their partners talked about coming developments that will definitely push GIS in a new direction over the upcoming year, for good or for worse.

Following are my own personal highlights from this year's UC. Please note that I am a .NET developer and self-proclaimed technologist, and am, therefore, totally geeky and focused on topics that are of interest to me. I didn't get much out of the "Everyone Loves/Needs/Uses GIS" sessions, the plenary talks, or the desktop productivity sessions, although overall it was nice, as a colleague of mine said, to hear others talking about GIS when having dinner at a random restaurant in the Gas Lamp District. I did, however, get a lot out of interacting with ESRI developers and the various vendors, and these are the moments  that I want to pass on in this write-up.

Monday, June 18th

Plenary

A good portion of the plenary fit into the "Everyone Loves/Needs/Uses GIS" category mentioned above. I'm glad to know how GIS is being used around the world, but as a GIS professional I don't have to be sold on the idea of GIS as an invaluable tool. The demonstrations and announcements at the end of the session caught most of the news, although unfortunately a lot of them fit into enhancements that should have either been included with ESRI's 9.2 releases (or 8.3, 9.0, or 9.1 in some cases) or addressed in service packs - that is, now that ESRI is actually releasing new features, not just bug fixes, in its patches.

Like I said, most of the highlights were covered in Jack's response to the UC Questions and Answers that was released last week, but here are a couple of developments that really caught my eye (note that I cover most of these, in-depth, in the commentary below and also that there are many more developments that didn't make it into the plenary that I cover in-depth below):

• More Transparency Within ESRI's Support System
• Release of ArcGIS Server JavaScript and REST APIs
• Enhancements to ArcGIS Server's Tile Generation Capabilities
• New Permissions Management Capabilities Added to ArcGIS Server
• Better Print Support for ArcGIS Server
• New Mobile Client for ArcGIS Server
• Support for PostgreSQL in ArcSDE 9.3

Tuesday, June 19th

ArcPad 7.1 - An Introduction

I've been working with the 7.1 beta for a couple of months now, and, because of this, was excited to hear a bit more from the ArcPad team about the upcoming release. First of all, I'll say that ArcPad is improving dramatically. The project was pretty stale for awhile, but seems to be picking up development momentum (which it really needs). The good news for ESRI is that there isn't (and hasn't been) much real competition out there in the GPS software arena, so they haven't lost much ground. Actually, it seems like they are - amazingly enough - still in front of the small group of GPS software vendors. Sure, many users love Terrasync and its easy-to-create forms, but most major organizations that use GIS are ESRI shops and need: 1) something that will fit into their overall GIS infrastructure (read: enterprise) strategy and 2) software that is released and supported by a company that they know will not go under in a year's time.

ArcPad 7.1 introduces a new extension for ArcGIS Desktop. They're calling it "ArcPad Data Manager" and, unlike ArcPad 6.x's desktop extension, it is installed (and, thankfully, un-installed) with ArcPad (so it is not installed when ArcGIS Desktop is installed). This new tool adds quite a bit of functionality to the check-out/check-in process, which is a process that, although it always seems to work well for me in 7.01, is always the most difficult part of working with ArcPad for users who don't have a lot of technical experience. I can't tell you how much time I've spent walking users through the check-out/check-in process (literally over and over), even after developing comprehensive step-by-step documentation that shows them exactly how to do it. I'm thinking, though, that this new data manager extension will solve many of these problems, as the status of checked out data are no longer stored locally on a computer. They are, rather, stored within the checked-out data themselves, which means that a user can check data out from one computer and then check it back in using another computer and it doesn't have to live in the same exact directory location to be successfully checked back in. On top of this, check-in can now be initiated from within ArcMap, allowing a user to browse to the data that need to be checked-in rather than having to make sure that the data that need to be checked-in live in the exact same place in the directory as the data that were checked-out. Although this may not sound earth-shattering, it is a very welcome enhancement that will likely save me hours of troubleshooting by telephone.

To describe the new ArcMap extension more fully: When the "ArcPad Data Manager" extension checks data out of a geodatabase, the data are now placed in a single xml file, with an .axf extension. This allows for all of the data to remain in a single file. The .axf format uses SQL Server Compact Edition to store its data, which allows for much more to be done with the data in ArcPad.

The biggest enhancement that this new format allows is the support of related data within ArcPad. We were shown a demonstration of creating a single geometry feature in ArcPad and then collecting attributes that lived in related tables. This is obviously a huge development for ArcPad, and is an issue that the ArcMobile team doesn't seem to have a clear solution for (more on this in the ArcGIS Mobile - The Road Ahead and Developing Applications with ArcGIS Server Mobile SDK sections below). The important thing to get across here is that these relationships are supported in the data (the .axf file), not in the forms. This should allow for easier transition in and out of a geodatabase without having to custom-design applications. Another important development is that the check-out and check-in tools will now be available in ArcToolbox, allowing for easy scripting of the check-out/check-in process with Python. I'm personally hoping that I'll be able to make the whole check-out/check-in process completely invisible to our organization's mobile users, as - like I mentioned before - this is the biggest problem that they run into when working with ArcPad today.

A couple of miscellaneous other enhancements that were discussed in the session, in no particular order:

• On check-out, you can now name the .apm that is created whatever you want.
• Data that are checked out can now be optionally encrypted and password-protected.
• Forms will no longer be overwritten on check-out, unless specified by the user. This means that customized forms can be built to support data collection and used without the user having to manually move and overwrite the form that is generated on check-out out of the geodatabase.
• If the geodatabase enforces unique identifiers, this will be supported in ArcPad, as well. As a sidenote, it should be easy to generate GUIDs with the .NET Compact Framework and SQL Server Compact Edition, so if you use GUIDs a lot, it should be simple to implement in the new ArcPad.
• New editing features introduced.
• Better GPS height support.
• The ability to capture features using a traverse method.
• Better support for cameras.
• Better search capabilities within ArcPad:
  • The ability to pre-configure query forms within ArcPad Application Builder Studio is now included.
  • A more intuitive search interface, including support for basic SQL.
• Support for read-only features.
• Enhancements to label tools.
• The ability to set a snapping tolerance, per layer, within ArcPad.

Another question that kept on coming up, over and over, at the UC centered on the seeming overlap between the ArcGIS Mobile and ArcPad products. Here's how it was described to me: ArcMobile is meant to serve as an easy-to-use client for ArcGIS Server. It can connect to servers in real-time and has some GPS functionality integrated. It does not, however, support many of the advanced features that ArcPad does, including support for connected devices (rangefinders, etc.) and will never have as much GPS support and as many advanced features as ArcPad. The ArcMobile team is really looking to appeal to a specific market, those who use ArcGIS Server and have a need to collect some basic data while in the field. To me, ArcPad really shines when there is a need to collect complex data into a geodatabase without having to custom design an application. ArcMobile seems to be better suited in cases where a very-targeted application that communicates with ArcGIS Server is needed and more flexibility is desired for developing on the client-side.

For me, I'm interested in both technologies. I love to have choices when hashing out the requirements of an application and deciding on a solution, and I truly think that my organization will be using both ArcMobile and ArcPad, depending on the needs of each specific project.

ArcGIS Server - The Road Ahead

There were tons of upcoming enhancements to ArcGIS Server 9.3 announced at the UC. Most of these, however, for me fall into the "should've been there at 9.2" category. I don't want to always be negative, especially when I think that ArcGIS Server is a great product that has improved dramatically over the last two service packs (even with some of the workarounds that are still required, yet-to-be-fixed bugs, and lack of cohesive and comprehensive documentation). ESRI needs to realize, though, that they can't continue to release beta products as runtime.

One glaring example of this is the documentation for ArcGIS Server 9.2; it basically didn't exist when 9.2 was first released and still lacks some critical pieces. What's the solution (other than trying to catch up on the documentation as soon as possible)? I've got a couple of ideas:

1. Loosen the release cycle, opening it up so that the different product teams can release when their product is actually ready. They've done this with ArcGIS Explorer (which is another product that should've held off on its initial release), and, because of this, the team is on a very aggressive release schedule and is able to quickly respond to bugs and feature requests.
2. Open up the beta program to a larger audience. There are obviously problems with ESRI's beta process when their initial releases don't perform as advertised. Opening up the beta to more real users could dramatically increase the amount of feedback coming back to ESRI, allowing them to truly test their product before initial release.

Anyway, I'm not going to solve all of the world's problems today, so enough about that and back to the topic at hand: upcoming developments in the ArcGIS Server product.

First of all, ESRI announced in the UC Q&A that they realized that ArcGIS Server 9.2 shipped without proper documentation (see my rant in the first paragraph of this section above), especially for developers. In this session - and at a few other times during the UC - they mentioned this again and made it clear that they are working hard to improve their online help and documentation. This is, of course, very welcome.

Another very welcome development is ESRI's decision to release JavaScript and REST APIs for ArcGIS Server at 9.3. This is another development that was first announced the week before the UC, and then again at the plenary, and then mentioned/discussed several times during the course of the conference. But I never got sick of hearing about it. Here are some more details:

• The REST API will be completely securable, just like any other service that is served by ArcGIS Server.
• The REST API supports querying, geocoding, and geoprocessing.
• The JavaScript API(s) will allow for easy development and deployment of lightweight web applications that consume both ArcGIS Server services and third party data services, like data consumed from ESRI ArcWeb Services, Google Maps, Mapquest, Microsoft Virtual Earth, and Yahoo! Maps.
• A token-based authentication system for JavaScript clients is also being released.

Further enhancements that are coming to ArcGIS Server 9.3 follow, in no particular order:

• Enhanced AJAX Support
  • Microsoft's ASP.NET 2.0 AJAX 1.0 extensions will be fully supported in the 9.3 release of ArcGIS Server. This includes support for Update Panels. With this addition, developers now have a choice as to which callback model they use. It will still be possible to use the current ASP.NET 2.0 callback implementation that is used in 9.2, but I am recommending that all of our applications migrate fully to ASP.NET AJAX, as it is a powerful, clean, and easy-to-use implementation. As an aside, as many probably already know, it is currently possible to use ASP.NET AJAX, but there are a lot of workarounds to get some of the ArcGIS Server controls (especially the map control) to work correctly when wrapped in an Update Panel.
  • Partial page rendering model supported (comes with ASP.NET 2.0 AJAX 1.0).
• New security management module will be included in ArcGIS Server Manager. This will allow administrators to set up access permissions for both services and applications using groups and users. I don't think that roles are supported, although I'm not positive about this. Of course, this is all currently doable in 9.2, but you have to manually configure NTFS permissions - if using Windows Integrated or Active Directory forms authentication - and manually make changes to the web.config. See the "Miscellaneous Questions/Answers and Other Tips that I Picked Up" section below for a little tidbit of helpful info that I picked up when speaking with the ArcGIS Server development team about configuring a web application and associated services to use Windows Integrated Authentication.
  • Ability to manage users per application and/or per service.
  • Can ensure that all communication with ArcGIS Server is encrypted (again, already doable in 9.2 but supposedly easier in 9.3).
  • As mentioned above, new token based authentication system for JavaScript clients.
  • Can work with Windows authentication or use forms authentication and store user information in SQL Server database.
• A big development in the open standards arena is that ESRI, with ArcGIS Server 9.3, will introduce support for GML, WCS (1.0 and 1.1), and WFS and improve its implementation of WMS. Unfortunately, though, for ArcGIS Desktop users the Data Interoperability Extension is required to access WFS services. To access WFS in ArcGIS Desktop, you must *install the Data Interoperability Extension (it doesn't have to be licensed/activated).
• At 9.3, Image Server will become an extension of ArcGIS Server. From what I was able to gather, this does not, however, move Image Server from Category B to Category A, so licensing for enterprise partners is not necessarily going to change with this release. That said, from talking with several ESRI employees, it does seem like they are considering eventually moving Image Server to Category A. An image service:
  • Can deliver data or pictures.
  • Is, from what they are saying, very efficient at serving WCS and WMS.
• Major enhancements to ArcGIS Server tile caches are coming at 9.3. As you can tell from earlier posts and comments that I've made on this subject, this is an absolute must for my organization. We have had major problems developing tile cache for many of our projects that cover large areas and need cache at large scales (e.g. half of the world at many scales, the largest being 1:1250). Actually, we haven't been successful in generating this particular cache, even using several servers for weeks at a time. Well, ESRI says that they've fixed many of the tile cache generation problems at 9.3. Here are some of the enhancements that will be coming out:
  • Ability to generate tiles that match the tile format for ArcWeb Services, Google Maps, and Microsoft Virtual Earth.
  • On-demand tile generation: This enhancement, if it works as advertised, will likely fix many of our problems, although it's not the perfect solution for our organization. This basically allows you to generate tile cache up front only for areas that you foresee users actually zooming into. For areas that may not get used as much (or at all), you don't have to generate tiles, but if a user zooms into them tiles are generated on-the-fly and then added to the existing cache for future users. This is a decent solution, although it's not always easy for us to predict where a user's interests lie, and I'm afraid that generation of tiles on the fly is going to be horrendously slow.
  • Generate tile cache based on extent of a feature: This is another *very welcome enhancement, as it should, theoretically, allow us to more effectively partition out and track our tile generation. One of the major problems that we currently have is that when cache generation fails for a large cache build operation, it is often difficult to discover exactly where it failed and then reinitiate the build from that point.
• Some user interface enhancements (for applications built using the wizard in ArcGIS Server Manager) coming at 9.3 include:
  • Zoom previous, zoom next buttons with another tool for toggling an overview map.
  • Map tips will be able to contain static text, images, links, and JavaScript.
  • A new print task will be available in ArcGIS Server Manager and will include the ability to work with map elements.
• Enhancements to the Web ADF will allow developers to take advantage of session state.

Some very exciting news above! I look forward to getting my hands on the new release.

Deploying and Using ArcGIS Explorer

This technical workshop with Bernie Szukalski presenting ArcGIS Explorer was one of the best of the conference for me, if only because it's always nice to hear from someone who is excited about the topic that they're presenting. I've played around quite a bit with ArcGIS Explorer and have been encouraging our organization's GIS users to use and deploy it since the last build (380) was released, but it was still great to see some of the tweaks and operations (tasks) that the interface supports. It was also good to hear a bit about the development environment surrounding ArcGIS Explorer and get some of my questions answered by those who work on the project.

The ArcGIS Explorer development team is not constricted to the standard ESRI release cycle, so they've already had four releases since the initial release with the overall 9.2 release and they plan on releasing a new build (with feature enhancements, feature additions, and bug fixes) every two months. The next release is expected within the next couple of weeks.

As this was more a "show me your product" type of session, I didn't get much in the way of technical news from it. I did, however, get more in the Extending ArcGIS Explorer Using the ArcGIS Explorer SDK session, which is covered below.

Wednesday, June 20th

How to Develop Rich, Web 2.0 Style Flash Mapping Applications Using ArcWeb Services

I wasn't expecting to get too much out of this technical workshop, as, I'll be honest, I haven't paid much attention to ArcWeb Services as it has really come of age over the last year or so. Well, I'll tell you sheepishly now that I was very pleasantly surprised. ArcWeb Services seems to be another valuable tool in the expanding web map industry, and I'll be experimenting with it more shortly.

One of the main reasons that I want to spend some time working with ArcWeb Services, other than the product itself, is that it was mentioned in this session that ArcWeb Services and ArcGIS Server will likely be merged at some point. There are many similarities between the two, including, with the upcoming 9.3 release of ArcGIS Server, the support of JavaScript and REST, and the merge seems pretty logical to me. It seems that most of what ESRI is releasing (ArcGIS Explorer, ArcGIS Online, ArcMobile, ArcGIS Web Services, etc.) and spending most of their money on are technologies that either currently fit into ArcGIS Server or will likely fit into ArcGIS Server in the near future. Is this bad news for ArcIMS? Well, ESRI says that they're going to continue to support it, but why would you want to stick with a product that wasn't being actively developed?

ArcWeb Services 2.0 is currently in beta 1 and can be accessed via the Flex or JavaScript APIs. Beta 2 is expected to be released at the end of July, and will include some major feature additions, including support for consumption of ArcGIS Server services, ArcIMS services, and GeoRSS. In the near future, ArcWeb Services will also support WFS, WMS, and more formats and the use of geoprocessing tasks.

Some benefits that I can definitely see to using ArcWeb Services and its APIs:

• Support for 13 projections, all available client-side.
• Access to loads of data for thematic mapping.
• The ability to upload your own data to ESRI servers and then consume them in an ArcWeb Services application.
• Support for vector printing - this is a biggie that ArcGIS Server doesn't seem to get.

To test drive ArcWeb Services yourself, go to the ArcWeb Services site and sign in with your ESRI global login. You will then be able to apply for a free 90 day trial which will give you access to the APIs and underlying data and services. After the 90-day trial, cost is calculated based on a per transaction model (somewhere around $1250 for 100,000 credits).

Here are some other resources that may be helpful in your evaluation:

• ArcWeb Explorer 2.0 Lab
• 2007 Developer Summit Presentations

Designing, Deploying, and Using Cached Map Services

I sneaked in on the last part of this technical workshop, as when I showed up initially there was a sign on the door that told me the session was full. Looks like others are having difficulty building cache too, huh (note that you can see some of my specific gripes about building tile cache in the ArcGIS Server - The Road Ahead section above)? We've had to learn most of the lessons the hard way, so I was hoping that this session would help us avoid further costly mistakes.

From what I was able to catch, it seemed like a pretty productive session, with lots of tips and insights coming from the ArcGIS Server team. I couldn't help but think, though, how much time, energy, and money ESRI could save their customers by getting these tips out in the form of online help or blog posts. Seriously, though, if ESRI is committed to helping their customers build and maintain an enterprise GIS, shouldn't they make their knowledge on how to make their software work correctly publicly and readily available?

Here are the tips that I caught in the short time that I was in the session:

• ArcGIS Online uses multiple tile servers, all of which have local .mxds that are accessing local data (stored in file geodatabases).
• Once you build a cache, you can get rid of your original data, as it's no longer needed or used by a service.
• If using the "generate cache" command, zoom into the extent that you want to generate cache for, set the max scale, and then start the cache build.
• There is a new blog post with a Python script to help in building cache coming out on the ArcGIS Server team blog this coming week.
• Enhancements to tile cache generation at 9.3 will not "break" existing caches, they will, however, allow you to build cache more efficiently.

Extending ArcGIS Explorer Using the ArcGIS Explorer SDK

This session had some pretty good demonstrations on creating custom tasks for ArcGIS Explorer included, although it was really more valuable as just a deeper introduction into what ArcGIS Explorer is meant to do and how it works. The session started out introducing the basics of an .nmf file, which is an xml file that ArcGIS Explorer uses to store everything. Being xml, .nmf files can easily be programmatically created and edited, and can contain everything from layers (including symbology, etc.) to ArcGIS Explorer tasks. They can basically contain as little or as much information as you want, and, since they are just xml, editing .nmf files is an easy way for users to create "custom" ArcGIS Explorer applications without any programming at all.

If you want to further extend an ArcGIS Explorer application, though, you are going to have to develop custom tasks. Tasks are designed to be very lightweight. They serve as a client-side interface to gather input from users, present this information back to them, and interact with other tasks, if needed. The bulk of a task's processing work gets done on the server.

To help developers design custom tasks, ESRI has released an API that allows developers to interact with ArcGIS Explorer. Note that the SDK is .NET only.

Here are some tips for those of you who are developing tasks for ArcGIS Explorer:

• As drawing lots of results takes time, set visible to "False" to begin. This dramatically improves the overall user experience.
• Currently, the polygon property does not have area. This is because of difficulties with WGS84 and 3D.
• ArcGIS Explorer returns hidden layers, so make sure that you check for these when iterating through layers.
• You can override "OnIdle" to bring real time updates of data. There are currently no events exposed in the API to do this, but it is possible.
• Result::Description does honor HTML tags, which means that you can use CSS, JavaScript, etc. in the description.

One more development that is coming with the next build of ArcGIS Explorer is exposure of OpenGL, which should allow for lots of exciting things to come.

Thursday, June 21st

ArcGIS Mobile - The Road Ahead

I wasn't able to make it to the 2007 ESRI Developer Summit, so I missed quite a bit of news on the ArcGIS Mobile front. Because of this, I was especially excited to hear more information about ESRI's plans for the SDK. Most of the session's time, however, was spent on the new ArcGIS Mobile client (it looks like they're calling it "ArcGIS Mobile Editor", although don't quote me on that). I was a little disappointed by this, but it was interesting to see some of the demo apps that they put together for the new client, as most of the functionality that was demonstrated is available now with the existing 9.2 SDK. After the session, I was told that there should not be any major changes to the SDK (other than additions) from 9.2 to 9.3, so there shouldn't be much work involved when migrating to 9.3.

As for the SDK, there are some enhancements coming at 9.3:

• Support for more base layers, including large, non-edit vector datasets and tiled ArcGIS Server map services.
• More street routing and navigation functionality exposed.
• An updated geometry and sketch model, including making it easier to create features with GPS (this can be done with the 9.2 SDK, but it is difficult).
• Better standalone and related tables integration. It was not clear how they are going to deal with related data, although I did ask.
• Support for Windows Mobile 6.0 and Linux Mobile Data Web Service.

Like I mentioned, the majority of the session was spent on the upcoming ArcGIS Mobile Editor. Here are some miscellaneous features that were discussed and demonstrated:

• Target release date: January 2008, along with the rest of the 9.3 release.
• The client is targeting Windows Mobile devices.
• Disconnected, occasionally connected, and always connected use cases will be supported.
• Will not support related tables in initial release.
• The team has really focused on usability and the user interface shows this. It is very simplistic, without any icons at all (it uses text in place of icons). It was interesting that they mentioned looking to the iPod for inspiration when building the user interface.
• Allows users to both collect and update features.
• Has full GPS editing capabilities.
• Can synchronize directly with ArcGIS Server with virtually any kind of connection - including a satellite phone connection, WiFi, CDMA/GSM, and wired connection - which allows for real-time tracking of either a GPS device (via the tracklog) or field work completed.
• Feature search functionality and ability to connect to web service to search, as well.
• Release of a new module for ArcGIS Server Manager which allows for development and deployment of ArcGIS Mobile services and applications.
• When a mobile service is accessed by ArcGIS Mobile Editor, geodatabase subtypes and domains are automatically brought with it onto the device.
• Uses local caching for "non-operational" data (data that are not being edited). These should not be sent over wireless, but, rather, loaded onto the device.
• Compressed vector layers are supported for static viewing.
• ArcGIS Online will soon have services specifically designed for consumption by ArcMobile applications, including: StreetMap, satellite imagery, and topopgraphic data.
• Wireless synchronization can be tweaked to fit specific cases (e.g. synchronize when a connection is available, synchronize when a WiFi hotspot is available, or automatic synchronization in real-time).

Managing Distributed Data with Geodatabase Replication

This session, although it covered quite a bit of material that was review for me, did help quite a bit. At my organization, we had a couple of problems early on with replication that have now been solved. Overall, from my experience geodatabase replication seems to work fairly well, and it sounds like there are quite a few improvements coming in 9.3 that will make it even more solid.

Again, I'm just going to list a few of the highlights from this session, as a lot of the session was just reviewing the fundamentals of using geodatabase replication:

• No version is needed on a simple child replica. Having a version will only decrease performance of the geodatabase and the replica.
• With the current implementation of geodatabase replication, metadata are copied over when the first replica operation is run. After that, metadata changes are not reflected in the replica. This may change in future releases, although the enhancement is not currently slated for the 9.3 release.
• There are tools that allow you to apply most kinds of schema changes across replicas. Applying schema changes is a two step process:
  1. Compare replica geodatabases directly in a connected environment.
  2. Export XML and then import to apply changes to geodatabase that needs the schema changes.
• Geodatabase replication can be integrated into larger application through APIs. There are two APIs, one for coarse-grained control (GeoDataServer object model), the other for more fine-grained control (Pre-ArcGIS 9.2 object model that works only on geodatabases that live on the Local Area Network).
• Enhancements coming at 9.3:
  • Replication will be extended to allow one way replication to file and personal geodatabases.
  • Better support for synchronization of large (greater than 1 GB) delta files.
  • Improved logging for easier troubleshooting of issues.

Lastly, at the end of the session it was announced that a whitepaper, titled "Managing Distributed Data" will be available on the ESRI Support site soon.

Developing Applications with ArcGIS Server Mobile SDK

This was another session that I didn't get a lot out of. Maybe I was just getting tired? Here are a few random notes:

Overview of building mobile applications for accessing ArcGIS Server services:

1. Build your geodatabase.
2. Author a mobile map.
3. Publish your map as a mobile service.
4. Design mobile application.
5. Deploy mobile solution.
6. Synchronize mobile solution.

Tips for developing mobile applications for accessing ArcGIS Server services:

• Use MapCache Generator tool on desktop to generate mobile map cache.
• Use Request_Completed event to monitor synchronization requests.
• Use the State and Notifications Broker API and be smart about how connected you are (Microsoft.WindowsMobile.Status).

Miscellaneous Questions/Answers and Other Tips that I Picked Up

So, as all of you who have been to the UC know, a lot of what you learn comes outside of the official workshops and technical sessions. The vendor area holds a plethora of resources in the form of technical expertise - especially for developers - that are available to help troubleshoot specific issues and field questions, and you can always learn something when talking to colleagues. Here are some miscellaneous anecdotes that I discovered during my time here in San Diego. Where appropriate, I've given credit to the originator of the information:

ArcGIS Explorer

• Can an .nmf be placed on a file server and not be downloaded/cached locally when a user opens it from his/her workstation?
  • It depends on the browser behavior. ESRI hasn't really looked into this, but they are going to and will come out with some guidance/recommendations on how to make various browsers handle .nmf files in this way. If your organization uses Internet Explorer, this can likely be set by group policy.
• Have any formal tests been done to help gauge the bandwidth cost of using ArcGIS Explorer on a corporate network?
  • No formal tests have been done, but because of the way that ArcGIS Explorer separates data into layers (which is a good thing, right?), it probably consumes more bandwidth than Google Earth. ArcGIS Explorer is designed to be a broadband application and will, therefore, not run with less than 100 kb/s bandwidth. For a good experience, there should be at least 200 kb/s of available bandwidth per user.

ArcGIS Server Performance

• Generating Tile Cache
  • Just a tidbit of information: to generate tile cache for the globes that are served through ArcGIS Online, ESRI used three quad CPU servers to build each of the caches. It took them two weeks to build each cache.
  • When generating tile cache for use in ArcGIS Explorer, it is possible (and recommended) to generate 2D cache and then associate it with a globe. Draping will happen on the fly, and this allows for cache to be generated much faster and with much less overall processing.
• Using Multiple DNS Entries to Serve ArcGIS Server Tile Cache
  • Lately, I've been working hard on improving performance of some of our web applications. One tried and true method that web developers live by is registering multiple DNS entries to serve out resources (CSS, Images, JavaScript, etc.), as this helps in getting around HTTP 1.1's two connections per-browser limit. I haven't been able to make this work for our tile cache yet, so I posed the question to one of ArcGIS Server's developers. His idea was to specify the names of the servers in the code and then use JavaScript to cycle through them on each image request. We then went over to discuss it with the product lead who told me that a blog post will be out in the next couple of weeks that will detail how to set this up.

ArcGIS Server Security

• At 9.2, it isn't possible to use Windows Integrated pass-through authentication from an ASP.NET ArcGIS Server application to a database server to, for example, limit edit access to a service based on an Active Directory logon. There are ways, however, to work around this and basically get the same thing. Note that what follows isn't pretty:
  • You have to place the service in a directory that has NTFS permissions set on it and make some modifications to the web.config. If a user that is not allowed access to the service goes to the web application, they will not see the service.
  • In this way, you can 1) create two different services that access the same SDE data (one for editing, one for viewing) and 2) restrict access to the services using NTFS permissions and modifications to the web.config. After doing this, the user will see only the service that they have access to in the application.
  • However, in addition to this you have to programmatically hide the "Edit" task that displays in the web application if a user does not have edit access to any services (and, if you want to make it even more complicated, you have to create and programmatically hide or display custom edit tasks per Active Directory group if you have multiple services that are editable by different groups of users).

Web Map Optimization (thanks Roland!)

• Cartographic representations of data draw faster in web maps than explicitly-defined symbology representations.
• It is good practice to convert .bmp to .emf before publishing maps. There is a style set, titled "ESRI Optimized Style File" that is available for use. This set contains styles that have been designed specifically for display on the web.
• Where possible, it is good practice to simplify your geometries before serving them through a web map application. For example, the "Simplify Polygon" command honors topologies while generalizing the geometry of a polygon feature class.
• When building a map that is going to be served through ArcGIS Server, define your scales initially and then group data that display at specific scales (scale-specific rendering) into group layers. ESRI has defined its standard scales, and loads their "StandardScales.txt" into each of their .mxd projects before doing anything else.

I was trying to overcome one of the shortfalls in ASP.NET 2.0's built-in controls today and stumbled on a gem: an extended GridView control that was built by Bilal Haidar.

The problem I ran into came up when I defined a fixed height for a GridView control. If you don't define a fixed height, the GridView dynamically changes its height when you page/sort, etc. This can obviously be a design problem, so I definitely like to avoid it. The thing is, if you go ahead and define a fixed height and then page to a view with only a couple of rows, the rows that do exist expand in height to fill up to the defined height. This can also be a major design problem.

Well, the search for a solution led me to XGrid.

First of all, and most importantly, this is not a completely rewritten control; it, rather, extends on the core capabilities of ASP.NET 2.0's GridView control. There are lots of custom Grid controls out there, including Telerik's AJAX-enabled r.a.d.grid and ComponentArt's Grid for ASP.net, but most of them are rewritten from scratch.

Here's a highlight [at least for me] of the added features; a complete list of the XGrid control's features is available at ASPAlliance.com:

• Single row and double row click events
• Context menu for each row
• GridView height fixed when number of rows is less than fixed height
• Built-in filter textbox

You can download the new control from the following link: http://authors.aspalliance.com/bhaidar/946.zip. To install, grab the CustomControls.dll from the bin directory, put it in your site's bin directory, and add the controls to your toolbox in Visual Studio.

Happy gridding!

For organizations that have adopted Microsoft Office as their standard office suite, working with Office formats (Access, Excel, and Word) in web applications is often a necessity. Users are familiar and comfortable with Office, and with added support for Excel spreadsheets in ArcGIS Desktop 9.2 exporting to Excel from database controls (GridView, DetailsView, etc.) is a great way to make your data more accessible to GIS users.

My group provides access to several organization-wide databases through our enterprise GIS intranet web portal, and we wanted to allow users to query out data and then export the data to an Excel spreadsheet. They would then be able to bring the spreadsheet into ArcMap and link it with their existing spatial data. Looking at the larger picture, this is a great - albeit not that elegant - way to tie disparate systems together and allow data developers to error-check their data by providing crosswalks between databases.

Although exporting the contents of an ASP.NET 2.0 GridView control is simple, I ran into a couple of issues right off the bat:

• With paging enabled on my GridView control (which is a necessity for our web application), when the contents of the GridView were rendered in Excel the page links at the bottom were preserved and passed into the spreadsheet.
• With sorting enabled on my GridView control (which is, again, a necessity for our web application), the column headers were passed as hyperlinks into the Excel spreadsheet. When they were clicked on an error message displayed.

Thankfully these problems were easily solved. As you'll see in the code below, the solution simply involved turning paging and sorting off temporarily and then turning it back on after the "Export to Excel" is completed.

Note a couple of caveats:

1. This will only work for exports of 65,000 or less records. Even though you can get up to 65,000 records returned, be aware that there is a performance hit on the server. It is a good idea to build the interface so that users have to first query out records before exporting to Excel.
2. I have tested this in Internet Explorer 7.0, Mozilla Firefox 2.0, and Opera 9.0 with both Office 2003 and Office 2007. When opening the exported .xls with Excel 2007, the following warning comes up:

When you click on the "Yes" button, the spreadsheet opens fine. This is not much of an issue for our organization, as we are still using Office 2003 and likely will be for at least another year. I found a reference to this problem in the ASP.NET forums, but was unable to find a solution. If anyone runs into a solution, please let me know.

With those caveats in mind, here's the code:

ASPX - Assuming that you are have a valid SqlDataSource set up:

<asp:Button ID="btnExport" runat="server" OnClick="btnExport_Click" Text="Export to Excel" /><br />

<br />

<asp:GridView ID="GridView1" runat="server" AllowPaging="True" AllowSorting="True"
   AutoGenerateColumns="False" CellPadding="4" DataSourceID="SqlDataSource1"
   ForeColor="#333333" PageSize="50" BorderColor="Black" BorderStyle="Solid" BorderWidth="1px">
     <FooterStyle BackColor="#5D7B9D" Font-Bold="True" ForeColor="White" />
     <Columns>
          <asp:BoundField DataField="DataField1" HeaderText="Data Field 1" SortExpression="DataField1" />
          <asp:BoundField DataField="DataField2" HeaderText="Data Field 2" SortExpression="DataField2" />
          <asp:BoundField DataField="DataField3" HeaderText="Data Field 3" SortExpression="DataField3" />
     </Columns>
     <RowStyle BackColor="#F7F6F3" ForeColor="#333333" />
     <EditRowStyle BackColor="#999999" />
     <SelectedRowStyle BackColor="#E2DED6" Font-Bold="True" ForeColor="#333333" />
     <PagerStyle BackColor="#284775" ForeColor="White" HorizontalAlign="Center" />
     <HeaderStyle BackColor="#5D7B9D" Font-Bold="True" ForeColor="White" />
     <AlternatingRowStyle BackColor="White" ForeColor="#284775" />
</asp:GridView>

Code-Behind:

public partial class ExporttoExcel : System.Web.UI.Page
{
    protected void btnExport_Click(object sender, EventArgs e)
    {
        GridView1.AllowPaging = false;
        GridView1.AllowSorting = false;
        GridView1.DataBind();

        Response.Clear();
        Response.AddHeader("content-disposition", "attachment;filename=ExcelSpreadsheet.xls");
        Response.Charset = "";
        Response.ContentType = "application/vnd.xls";

        StringWriter stringWrite = new StringWriter();
        HtmlTextWriter htmlWrite = new HtmlTextWriter(stringWrite);

        GridView1.RenderControl(htmlWrite);

        Response.Write(stringWrite.ToString());
        Response.End();

        GridView1.AllowPaging = true;
        GridView1.AllowSorting = true;
        GridView1.DataBind();
    }
    public override void VerifyRenderingInServerForm(Control control)
    {

    }
}

The code is pretty straightforward. On the btnExport_Click event, paging and sorting are turned off on GridView1. Next, the content type is set and passed to the browser as HTML and the download is initiated. After the transfer is completed, paging and sorting are turned back on. Finally, VerifyRenderingInServerForm is overridden, confirming that an HtmlForm control is rendered for the server control at run time.

Hope this helps...

There's lots of guidance available on the web for encrypting configuration sections of your ASP.NET 2 application's web.config, but here's a quick reference:

You can store your key in either a machine store or a user store. If your application runs on a dedicated server or if you want to share the sensitive information between multiple applications, use the machine store. If your application runs in a shared hosting environment or you want to make sure that no other applications are able to access the sensitive information, use a user store. Note, however, that more complexity is introduced when using a user store. For more information, see "How To: Encrypt Configuration Sections in ASP.NET 2.0 Using DPAPI", available here: http://msdn2.microsoft.com/en-us/library/ms998280.aspx.

Note that some sections cannot be encrypted using ASP.NET 2's encryption mechanism:

• <processModel>
• <runtime>
• <mscorlib>
• <startup>
• <system.runtime.remoting>
• <configProtectedData>
• <satelliteassemblies>
• <cryptographySettings>
• <cryptoNameMapping>
• <cryptoClasses>

They can, however, be encrypted using the method that was around for ASP.NET 1.0 and 1.1 (and is still around for ASP.NET 2). More information is available in Microsoft Knowledge Base article 329250, available here: http://support.microsoft.com/kb/329290.

To encrypt a connectionStrings section using the machine store:

aspnet_regiis -pe "connectionStrings" -app "/YourApplication" -prov "DataProtectionConfigurationProvider"

To encrypt a connectionStrings section using the user store:

aspnet_regiis -pe "connectionStrings" -app "/YourApplication" -prov "MyUserDataProtectionConfigurationProvider"

In these example, aspnet_regiis.exe is located at %System%\WINDOWS\Microsoft.NET\Framework\<versionnumber>. The -pe switch assumes that you're using IIS and that YourApplication is the name of a Virtual Directory on your server. If you're using Visual Studio 2005's development web server, you can use the -pef switch to specify the physical directory location of your web.config.

To decrypt a connectionStrings section using the machine store or the user store:

aspnet_regiis -pd "connectionStrings" -app "/YourApplication"

Again, the -pd switch assumes that you're using IIS and that YourApplication is the name of a Virtual Directory on your server. If you're using Visual Studio 2005's development web server, you can use the -pdf switch to specify the physical directory location of your web.config.

Visual Studio and I have a love hate relationship. I love working in it, but why does it have to hog all of my system resources? I've got a decent system (3GHz Pentium 4, 3 GB of RAM, etc.), but lately I've been running into major issues: 5 minute freeze-ups, devenv.exe using ~200 Mb of memory (which seems like a lot to me, but I found posts from other users saying that it used over 500 Mb on their systems), etc. Doing a Google search for "devenv.exe hogging resources" shows that many others are experiencing the same type of problems. The search also, unfortunately, reveals that there really isn't a great solution for the problem. You can always disable add-ons, of course, but this doesn't always fix the problem and sometimes just isn't an option.

Well, I ran into an interesting and bizarre workaround: when you see that devenv.exe is getting out of control and funky minimize the Visual Studio window(s) and then restore it/them. This takes devenv.exe's memory usage down to a more reasonable level. Watch out, though, as it will likely start to slowly creep up again.

The ultimate inefficient workaround that I'm currently implementing is downloading the next version of Visual Studio, currently codenamed "Orcas". I'm going to run it in a virtual machine on my desktop. I've been meaning to do this for awhile, as it's got a lot of features that I'm really interested in trying out, including full support for C# 3.0 and integration of the Language Integrated Query (LINQ) project, but this VS 2005 performance issue is pushing me towards this workaround a little faster than I might have gotten there otherwise.

Anyone else have tips on optimizing the Visual Studio environment?