At DrDoctor we are broadly speaking following the Agile Scrum methodology; we do Sprint planning, work on user stories, hold retrospectives etc… Where are break away from the traditional ‘sprint’ methodology is how we go about getting stories “accepted” and released. Traditionally in Scrum you demo all the stories at the end of the sprint in the Sprint Review meeting, whereas we tend to demo stories (and have them accepted by the PO) as they are completed and then we aim to have accepted stories released to production within 3 days.
At DrDoctor we do a lot with SMS, we let patients reschedule appointments, book appointments when they move to the top of a waiting list and we provide patients with the ability to move their appointment forward by sending them lists of slots as capacity becomes available. In all of this we use workflows to represent what is going on between the patient’s mobile phone and our system. We call them workflows but really they are just state machines built on top of the excellent .
Over the past couple of weeks we’ve noticed that our Reporting service is terribly slow. The above chart is part of our APM (Application Performance Monitoring) solution built on top of MiniProfiler and Splunk (see this blog post for more details). Our reporting service works by processing messages which are queued up by various other services onto a queue in RabbitMQ. What this chart is showing is that the reporting service is working at a very constant and slow pace.
Last week I deployed a new Api hosted as an Azure WebApp, we wanted some reporting events to come out of it into our Splunk instance so we could keep an eye on whether it is working as expected. I started off by using the Splunk C# SDK as it looked nice and easy to add into our app. A very trivial example of using the Splunk SDK would look like this:
At DrDoctor we are slowly adopting Splunk as our central reporting repository. We already have most of our application specific events going into it and we are already seeing some great benefits. In this post I’m going to show the various steps I went through to get our log4net files being ingested in a useful format. Monitoring a file is easy, extracting useful fields is sometimes a challenge especially with log files.