We have had a few niggles with calls this week. On Monday we had some outgoing calls dropping after 30 seconds. This was resolved quickly with a change of carrier. We have now made a more permanent fix.
We have had some issues this morning for a couple of minutes where calls could not be established. Calls in progress should not have been affected.
Also, some customers have reported some more vague problems over a longer period of time. There is a well known effect with any phone system changes that people look at every dropped call, every mis-dial, every race condition, and report problems even though they are quite normal. However, we are concerned that these reports may be genuine, and would ask customers that have any concrete examples to email support with a date and time and number and description so we can investigate.
Something not right - code being reverted now.
OK, the code had a minor change to fix a temporary fix done at the start of the week.
Somehow this has made things stable, and the code has been reverted to previous version while we get to the bottom of it.
Looking stable now.
In general the problem has related to calls being set up and not calls in progress. We are monitoring the situation closely.
OK, we have concluded that even with previous and very stable code there is still a problem, so we are checking packet logs to confirm what is breaking things now.
Looking stable now.
We have seen a few issues later in the day but extra diagnostics are showing up a likely cause which is being addressed now.
OK, a slight post mortem to follow.