Personify Data Services Diagnostics

Personify recommends that you turn off all logs by default and only turn them on in case of system failures.

PDS comes with comprehensive diagnostics implementation. It is integrated with the open source logging platform called NLog. NLog is a free logging platform for .NET, Silverlight, and Windows Phone with rich log routing and management capabilities. It makes it easy to produce and manage high-quality logs for your application regardless of its size or complexity. For more information, please visit http://nlog-project.org.

The following are traced as part of diagnosis:

· Business Assemblies – Layers loaded along with priorities.

· Layer Map – Classes/Service Operations taken into account based on layer/priority configuration.

· OData URI – request URI received by the service (this has to adhere to our supported operators covered in OData URI Conventions and Query Options section).

· Evaluated SQL – SELECT statement generated by EF from LINQ expression (this is not going to be executed directly against database).

· PersonifyApiParams – The structure of data (along with filters) requested by client/consumer (to APIs). This is parsed from SELECT statement explained in previous bullet.

· Service Operations – Every Service Operation requested for execution (including Create/Save) is logged.

· Serialization/Deserialization issues – Service Operations receive XML and are de-serialized to entities at server. If the data is not appropriate for serialization/de-serialization, it will raise an error and place entire data into log file.

Note:

· It is important to make sure that the column order (and number of columns) matches with the columns in SELECT statement. If it does not, it could be a bug. You may need to send diagnosis information (log file) in order to reproduce/fix the issues.

· Validation Issues are thrown as errors to the clients/consumers (these are not included in diagnosis)

· If you are not diagnosing the service (and especially in production environment), it is not recommended to enable the diagnostics, as it writes a lot of information to log, which would decrease the performance of service responses.

Enabling Diagnostics (Logging)

In order to enable logging, make sure you have the following entries in Web.Config:

…

</configSections>

<nlog>

</nlog>

Here, [PATH_TO_NLOG_DS_CONFIG_FILE] is the name of logging configuration file. You can pick any arbitrary name for the file such as “NLog.config”.

Configuring Logging

After you enable logging in the Web.Config file, the next step is to determine what to log, how much information to log, and where to log – file system or database, etc. All these can be configured in a separate configuration file that you have specified in the previous step - [PATH_TO_NLOG_DS_CONFIG_FILE]. Typically the file name would be “NLog.config”, residing in the same folder as “Web.Config”.

Personify has provided a template file called “Template_NLogDS.config” as part of the installation package. You may rename the template file to the name that matches your entry [PATH_TO_NLOG_DS_CONFIG_FILE], typically “NLog.config”.

Additional detailed documentation is available at: http://nlog-project.org.

Logging Levels

“Level” determines what to log. NLog provide six different levels of logging, as described below:

1. Trace. The most verbose logging. Used for debugging messages, that is of interest to developers during troubleshooting. Can be used to log intermediate output results of queries, methods etc. Should be turned off in production.

2. Debug. As with Trace, is used for debugging purposes, with developers as consumers. The difference with Trace is very minimal; a lot of times they are just synonyms.

3. Info. Used for logging information that can be of interest to operational team, as well as important information for developers. Examples: various configuration parameters that are currently being used, name of DS servers, various statements about inter-process requests (such as format of queries to data services), etc.

4. Warn. This is something that most likely requires attention. It does not necessarily mean an error, but it can contain important information that might give you a clue as to why the system does not behave as you expect it to. May contain validation issues, business warnings, unexpected inputs/outputs, etc.

5. Error. This level is used for specifying errors when system was unable to perform its business function for whatever reason. A lot of time it means an exception was thrown; however, not every exception is an error – for example, validation exceptions may be just warnings or not errors at all; here, only exceptions that forced the system to fail in performing its business function are treated as errors. This level does not necessarily mean that immediate human actions are required; as an example, occasional invalid data input for Data Service is an error; but that error does not affect other users at all, so this error message may be analyzed later.

6. Fatal. This means that serious system crash has occurred, and the system will not continue to operate. This level definitely requires human activities to return the system back to its normal state.

It is very important to understand that more the details you log, the lower the performance you get. You may decide what level you want to log in your development environment versus your production environment based on your best judgment and experience.

Filtering

NLog also provides an option to include or exclude certain classes by specifying the filters, as shown below:

<rules>

</filters>

</logger>

</rules>

Implementing Logging

While writing your custom service operations, it is strongly recommended that you add code log the maximum details. In order to implement logging in the service operations, declare a class level static variable, as shown below:

private static readonly Logger _logger = LogManager.GetCurrentClassLogger();

Then, use the variable _logger to write the details using one of the six levels, as shown below, depending upon the context:

_logger.Trace("My Message");

_logger.Debug("My Message");

_logger.Info("My Message");

_logger.Warn("My Message");

_logger.Error("My Message");

_logger.Fatal("My Message");

Diagnosing an Error

Before we start on diagnosing, you will need to make sure that the following checklist is passed:

· At service level

o Properly configured “config.xml”

§ Application server settings

§ Seat Information

§ API/Form libraries (if customized)

o Properly configured “web.config” (refer “Understanding Configuration” for more information)

§ PersonifyBocState section configured to session state database

§ SessionState with “sessionIDManagerType”

§ PersonifyDataServicesLayer configured with respect to layers (customization)

§ AppSettings, DbProviderFactories, ConnectionStrings, authentication, membership and modules

o All Personify API, Custom, log4net, PDS core/model/business/custom dlls in “bin” folder (should reflect based on layer configuration)

o Service information related XML files in “bin” folder

§ Personify.DataServices.EF.Model.Base.xml – base delivery

§ Your custom service information XML files (generated using WSD)

· At client/consumer level

o Service URL – to connect to

o Authentication information – correct userid/password

o Go to http://servername/websitename/servicename.svc$metadata - to give you all the entities/service operations (entire model) you created using WSD.

§ Ensure that you are looking at correct information.

§ If this does not go through, your schema/model may not be correct

If all of the above are passed, then the configuration should be working correctly. The next step is to have all the tools (and checklist) in place for diagnosis:

· Enable logging (covered in previous section)

· Enable debugging (in web.config)

· Record the traffic using “Fiddler” (free HTTP traffic monitoring tool available at http://www.fiddler2.com)

o “Fiddler” does not monitor traffic on “localhost”. Please make sure that your http requests are reached with either server/machine name or IP address in the service URL.

o You can select switch to RAW mode (for both request/response) in “inspection” tab for a particular “request” (selected from left pane) to understand the communication in detail.

· Restart IIS (or kill the worker process)

· Start consuming the service

o the diagnosis will be recorded in log file and http traffic gets recorded through Fiddler

· Look into log file and Fiddler for complete information

You can always debug into a “Service Operation” by attaching the (business) project to the worker process directly.

Reporting an Issue

A technical consultation (to resolve an issue) requires all of the following to be provided as part of your request:

· Service Schema/model file (generated through WSD)

· Log file

· Communication between client/consumer and server – HTTP traffic recorded through Fiddler (or any other http monitoring tool)

· Config.xml

· Web.config

· Custom TIMSS DLLs and Custom PDS Business DLLs (if any)

· Service Operation Source code (in the case of custom Service Operation)

· Consuming Source Code (if other than OData URI access)

· Environment information

· Complete description (or video) about the issue