We’re always trying to improve and refine the judging system of the International Chocolate Awards, based on our experience as we run more competitions, on the feedback we receive from judges and entrants and from the results of our annual review surveys.<\/p>\n
2013 sees a number of improvements, particularly for the selection round. We’ve also been refining and formalising the Grand Jury process to ensure that it’s is as fair as possible and reflects the Main round scores, while allowing the Grand Jury to work in an effective way as the final arbiters of technical issues and the competition prizes.<\/p>\n
One of our aims with the International Chocolate Awards is to help support upcoming chocolate companies and to help them to improve through constructive feedback. It\u2019s difficult though, when hundreds of samples are being judged in some of the larger competitions, to find ways to capture the opinions and notes of our judges while they are tasting.<\/p>\n
We always try to limit the number of samples tried in our main two-hour judging sessions to a maximum of 15-20 samples per judge \u2013 with around 15 to 18 being the ideal. Even so, it would be difficult for any judge to keep writing positive and useful comments about each sample, especially if they are not technically trained in every area of chocolate production and tasting. (We aim to include a wide range of judges with good palates, including sommeliers, chefs, journalists and others, so not all of our judges are chocolate specialists.)<\/p>\n
Since our first competition, we have given our judges a series of pre-written statements identifying a range of positive and negative qualities about each sample, so the judges can quickly circle the letter corresponding to each statement. This lets us reliably input the statements into our judging system and then the feedback is (anonymously) output through the secure part of our website for each entrant to review.<\/p>\n
In 2012 this system worked well for products that made it through to the Main judging round. \u00a0Each judge is given one judging form for each product they try, which includes the sample scoring section and also space for an extensive list of feedback statements. We feel that this gave quite comprehensive positive and negative feedback, which is far more controlled and far beyond any other competition.<\/p>\n
However, if a product was unlucky enough to not make it through to the Main round of judging, we gave only limited feedback in broad areas where the product either had problems or in areas where it could improve. An example of the areas for filled chocolates would be \u2018Execution, Formulation, Interpretation and Taste\u2019. So an entry might be rejected for \u2018Formulation\u2019 and \u2018Taste\u2019, but have \u2018Execution\u2019 and \u2018Interpretation\u2019 marked as areas of potential.<\/p>\n
We kept things very simple, because the Selection round is meant to be a quick-fire round judged by more experienced judges. The aim is to reduce the numbers before we get to the Main round judging in each competition. This is always a trade-off \u2013 we want to give each sample the best possible chance and the maximum judging time, so it\u2019s important that judges are not overwhelmed by the number of samples in the Main round.<\/p>\n
During Selection, entries are rejected if they have technical flaws, contamination, are spoiled or are very unlikely to have a chance of winning. If there is any doubt for an individual judge, they can vote \u2018Further judging\u2019, so the sample can go through. Each sample gets a single line on a form for up to 18 samples, with space just to mention the criteria areas mentioned above.<\/p>\n
We\u2019ve recognised though that this limited level of feedback might not be terribly useful, especially if you are a new or aspiring company and most or all of your entries didn\u2019t get past Selection. This was one of the points we covered in our 2013 Entrant Survey, and your feedback on our feedback was that we could do better!<\/p>\n
We\u2019re always trying to improve our system, so we\u2019ve come up with an improved Selection form for 2013 to address these issues.<\/p>\n
For the main competitions in 2013, we have a new Selection form in use. There\u2019s still one line per entry on each judge\u2019s form and the same votes for \u2018Reject\u2019, \u2018Further judging\u2019 and \u2018Award contender\u2019.<\/p>\n
For 2013 though, we\u2019ve added an extensive list of technical reasons listing possible problems with a product. We\u2019ve also added a new rule \u2013 no Selection round judge may reject a sample without giving a reason<\/i><\/b>. This is done by simply marking the relevant letter in a box on each line. This also takes away any temptation for a judge to skip over giving any feedback for a particular sample.<\/p>\n
We\u2019ve actually allowed space for up to three reasons, but there must be at least one. (We feel three is more than enough and gives an entrant good guidance on what needs improving to create a great product.) The possible reasons are tailored to each class of product \u2013 plain bars, milk bars, flavoured bars, filled chocolates and spreads.<\/p>\n
We\u2019ve also included a short but improved list of \u2018Areas of Potential\u2019. These are less important, as a product with good potential will go through to the Main round, but we are still encouraging Selection judges to add these, and if they vote \u2018Further judging\u2019, they must give at least one ‘Areas of Potential’ reason.<\/p>\n
We feel confident that the new version of the Selection form will give a much more useful level of feedback for this year and that it also gives the Selection judges much better guidance on the exact reasons for why they might reject an entry.<\/p>\n
Our judging system is completely open and transparent and all the forms can be downloaded from our Judges Systems and Forms page<\/a>. (We do maintain copyright on the forms though and ask that they are not used without permission.)<\/p>\n (Note: due to time constraints we were only able to implement this system in time for the European Semi-final, the Israeli and Italian competitions for 2013 used the previous system. All future competitions will use the new system, which we will continue to refine and improve.)<\/p>\n The Grand Jury round has been one of the areas that has had the most development since we began the International Chocolate Awards at the start of 2012 in Italy. Our Grand Jury members from last year will remember the 12.30am finish at the first European Semi-final in May – to get through everything we were still tasting and (sometimes hotly) debating the merits of each entry well past the time allotted for our communal end of Awards celebration meal.<\/p>\n We\u2019ve learnt a lot since then and the Grand Jury session is now very structured and coordinated with its own set of rules.<\/p>\n First we determine which of the highest scoring entries from the main round are contenders for Gold and Silver awards or if not, can be considered for Nominations to compete in the World Final. This is done be establishing a statistical \u2018break point\u2019 based on analysis of the Main round scores.<\/p>\n Each Grand Juror is then given a form for the current category being considered that lists the entries in score order, the scores and a column to vote in.<\/p>\n The potential winners are then presented with a break down of the scores and positions For the first time in the competition, discussion while tasting is encouraged. This is to be sure that any technical points or problems are fully identified. In larger categories, each product will be presented by a Grand Juror who will act as its advocate, pointing out its strong points or any problems, which could be technical or relating to the qualification rules or categories.<\/p>\n As with the rest of the competition, only the judges\u2019 anonymised description is used, the tasting is still completely blind until the end of the competition.<\/p>\n On a side note, we accept that it\u2019s inevitable that some judges will always recognise some products, but we believe credible judges will not be swayed by this and we always try to include a range of Grand Jurors from different countries and experiences. Knowledge of the subject is essential, and this inevitably – and should – mean some recognition of products. This is especially true when it comes to plain origin bars \u2013 any decent taster should be able to recognise different cacao origins and manufacturing styles right away.<\/p>\n Once the tasting is finished, the Grand Jury votes in private. The votes are then collated and averaged and the prizes are proposed, still blind at this point. Ideally this is done during the Grand Jury session, but occasionally this is not possible and final details have to be ironed out by email. (Or if the prize announcements are delayed for a later ceremony.)<\/p>\n It\u2019s important that the Main round scores are respected, given all the hard work put in and the range of judges present at the Main judging. So the Grand Jury is not allowed to \u2018move\u2019 the medal position, as determined by the Main round scores, by more than one place unless there is a technical reason for doing so. So a Gold can go to a Silver by voting, but not down to a Nomination or no prize, and the same applies in the other direction.<\/p>\n Grand Jury members may not vote an entry up or down just because they personally like or dislike it (or recognise the product).<\/p>\n The Main round scores can only be overridden by more than one place if a breach of rules is found or if a technical flaw is found. Examples of this could include: mislabeling, ingredient problems, mould or other off notes detected in plain bars, poor execution not noticed in main judging, artificial flavouring detected, use of poor or off tasting chocolate, being overly sweet or bland. These are just a few examples.<\/p>\n Ideally these types of problems will all have been caught at earlier stages, but the Grand Jury round is where full consideration of the top entries can be made.<\/p>\n The Grand Jury\u2019s role is to make a thorough examination of the potential winners and to ensure that all prize winners live up to the high standards set by the International Chocolate Awards.<\/p>\n So to sum up the improvements we\u2019ve made at the end of 2012 and early 2013:<\/p>\nGrand Jury<\/h2>\n
Blind tasting<\/h3>\n
Voting<\/h3>\n
Grand Jury rule changes<\/h3>\n
\n