Wednesday 2 November 2011

Total Daily Carbon Intensity in the UK - Plot and Prediction Analysis.

I try to analyse the data of Carbon Intensity in the UK. By getting the total daily carbon intensity in the UK, we can see the shape of the graph below (see figure 1).

Figure 1. Actual Total Daily Carbon Intensity in the UK (from 27 June 2011 to 27 September 2011)
As we can see in Figure 1, the carbon intensity during the weekday is apparently higher than the carbon intensity during the weekend.

After that, I apply a Single GPs prediction for the total daily carbon intensity in the UK. The initial training data set is the first 4 weeks (28 days), the predictive period starts from 29 to 92. In this prediction, I only apply one-day ahead prediction. For example, if we want to predict the carbon intensity for the day of 35, then I would consider the training data set is from 1 to 34. The result is in figure 2 below.

Figure 2. Prediction of Daily Total Carbon Intensity
The Mean Square Error (MSE) for this period of 64 days is 1944838. For more detail, Figure 3 shows the MSE for individual days during the predictive period.

Figure 3. MSE for Total Daily Carbon Intensity during the predictive period.

Let's look into further detail by predict Carbon Intensity for a day ahead, however we predict every half-an-hours instead of the total daily value. Previously, I only used the constant hyperparameter because it is time-consuming. This time, I use the initial training data set of 28 days, started from 27 June 2011. The predictive period is from the day of 29th to 58th. The hyperparameter is iterately trained everyday during the predictive period. Having waited a few hours (approximately 4 hours), Figue 4 below shows the result.

Figure 4. Single GPs on Carbon Intensity oneday ahead in the UK (with trained hyperparameters), started from 27 June 2011.
The MSE for this 30 days predictive period is 1000, which is very much improved. In addition, Figure 5.0 shows the MSE for this period in detail.

Figure 5.0. MSE of Single GPs on Carbon Intensity One Day Ahead, Every Half An Hour prediction.

Furthermore, we analyse an energy consumption of some real users. First of all, I plot the total daily of user's energy usage. These figures can be seen as following:

Figure 6.1. Daily Total Usage of User 1.
Figure 6.2. Daily Total Usage of User 2.
Figure 6.3. Daily Total Usage of User 3.
Figure 6.4. Daily Total Usage of User 4.
Figure 6.5. Daily Total Usage of User 5.

By using the same covariance function, which applied in Carbon Intensity prediction, I have tried to run some prediction. However, the result looks really bad. The covariance function for user's consumption has to be much different, which I still have not found out yet. Moreover, the resolution for the user's usage is every two minute, which is quite high. It typically takes much time to run the file and wait for the result. Particularly, when the training hyperparameter is applied, the waiting time could be take for a few hours.

I might need to reduce the resolution for the data of user's usage to do more test on GPs prediction.

2 comments:

  1. Thank you for this update Henry.

    Based on your explanation yesterday, I was expecting that the predictor would be better at spotting the different patterns between weekdays and week-ends. Why do you think it is not working very well?

    Did you train the predictor on the daily average data set?

    I look forward to see the rest of the analysis we discussed yesterday.

    ReplyDelete
  2. Hi Henry,

    It is not easy to comment on so many things on the same post..

    > the carbon intensity during the weekday is
    > apparently higher than the carbon intensity
    > during the weekend
    This is not a surprise: we know very well that commercia and industrial consumptions go down in the week-end, and that's why you (Sasan) used a combination of periodic functions for this prediction.

    What would be good to know more about is the fluctuation. For example why is it that on days 56, 70 and 77 the consumption is so different from other sundays?
    Gopal, do you have any pointers about this?

    > The MSE for this 30 days predictive period is 1000,
    > which is very much improved.
    Ok. Good that it is improved, but what is the target here?
    I think that now it is a good time for you to start writing about the ways you want to use these predictions.
    What I have in mind is that your prediction needs to be "good enough" for what you want to do with it. This should give us some context for
    these MSE errors, which I find otherwise a bit abstract..

    So, please write about how you want to use these predictions. We talked about it a bit, and we will probably talk more about it on Friday, but it would be good for you to put ideas in writing.

    > In addition, Figure 5.0 shows the
    > MSE for this period in detail.
    I was hoping to see that the more training data the smaller the error.. however, this does not appear to be the case from this figure.
    Can you comment on that, please?

    > the result looks really bad
    Are you sure it makes sense to run a prediction on this data set?
    I am a bit doubtful.
    Could try to predict more abstracted values?
    For example, could you predict the total amount of energy each user will consume in the next 1, 2 or 7 days based on their consumption history?
    I would imagine that could give more interesting and useful results.

    > I might need to reduce the resolution for the
    > data of user's usage to do more test on GPs prediction.

    Did you try that?
    How did it go?


    > Particularly, when the training hyperparameter
    > is applied, the waiting time could be take for a few hours.
    Why do you think it is so slow?
    Can you think of anything that would make things faster?
    How about randomised training?

    Enrico

    ReplyDelete