Wednesday, December 21, 2011

Bug Prediction

Recently I've read a very interesting post about Bug Prediction on Google Engineering Tools blog and decided to inspect FreeBSD source tree in the same manner.

The question is how to determine if the given commit is a bug fix. I decided to start with a security fixes first and filtered commit messages by "Security:" tag in them. The result is the following:



FilenameScore
crypto/openssl/ssl/s3_srvr.c1.12807360699
crypto/openssl/ssl/s3_clnt.c1.12546730725
crypto/openssl/openssl.spec0.914805209201
crypto/openssl/NEWS0.914805209201
crypto/openssl/CHANGES0.914805209201
crypto/openssl/README0.914805209201
crypto/openssl/crypto/opensslv.h0.914805209201
crypto/openssl/Makefile0.914805209201
crypto/openssl/FAQ0.914805209201
crypto/openssl/apps/speed.c0.707204148366
crypto/openssl/ssl/t1_lib.c0.696806056187
crypto/openssl/ssl/s3_lib.c0.663083892748
crypto/openssl/crypto/md32_common.h0.578509821664
crypto/openssl/crypto/ocsp/ocsp_prn.c0.578509821664
crypto/openssl/engines/e_chil.c0.578509821664
crypto/openssl/ssl/d1_pkt.c0.51304099663
crypto/openssl/crypto/pqueue/pqueue.c0.51304099663
crypto/openssl/ssl/d1_both.c0.51304099663
contrib/ntp/ntpd/ntp_crypto.c0.505424173077
crypto/openssl/crypto/asn1/tasn_dec.c0.482876843456
sys/compat/linux/linux_socket.c0.477720395098
usr.bin/compress/zopen.c0.462407652214
usr.bin/gzip/zuncompress.c0.462407652214
sys/kern/uipc_usrreq.c0.462407652214
crypto/openssl/ssl/ssltest.c0.460574251046
crypto/openssl/ssl/s2_srvr.c0.460574251046
crypto/openssl/apps/x509.c0.460574251046
crypto/openssl/ssl/s3_pkt.c0.456096100304
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_ioctl.c0.44243544619
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zvol.c0.44243544619


There results appeared not very interesting though. This kind of matches security advisories, but given that there are not very much of them (luckily!), it's not very representative.

I also tried to run the script with "[\t ]+[Bb]ug" regular expression instead of "Security:", and got the following result:

FilenameScore
sys/netinet/sctp_output.c7.66251938944
sys/netinet/sctp_pcb.c7.2065885975
sys/netinet/sctputil.c6.80461378901
sys/dev/bge/if_bge.c6.50257671046
sys/netinet/sctp_input.c5.62524237411
sys/netinet/sctp_indata.c5.25336437345
sys/netinet/sctp_usrreq.c5.11155404498
sys/netinet/sctp_timer.c3.67543247047
sys/netinet/sctp_structs.h2.99517512907
sys/conf/files2.89041302648
sys/netinet/sctp_asconf.c2.79978400331
sys/netinet/sctp_constants.h2.79452724525
sys/dev/bge/if_bgereg.h2.62852430236
sys/dev/msk/if_msk.c2.31771522145
sys/netinet/sctputil.h2.19989952769
contrib/bind9/CHANGES2.15204576684
contrib/bind9/version2.15204576684
sys/kern/kern_fork.c2.12687393752
sys/netinet/in.c2.09501338006
sys/netinet/sctp_sysctl.c2.06560623664
sys/netinet6/sctp6_usrreq.c1.92839801197
sys/netinet/sctp_var.h1.91990472478
sys/conf/NOTES1.79166008565
sys/dev/mxge/if_mxge.c1.74628407608
contrib/bind9/bin/named/query.c1.74336559317
contrib/bind9/lib/dns/rbtdb.c1.74336559317
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c1.7287195258
sys/i386/xen/pmap.c1.68724079063
sys/powerpc/aim/mmu_oea64.c1.68644933665
contrib/bind9/lib/dns/validator.c1.6676964412


This is somewhat more interesting result. It shows that most 'dangerous' sits in SCTP implementation, NIC drivers (bge, msk, mxge) and also some bits in zfs and bind. I wonder how accurate is that?

The script is available here. It's designed to use git repository. I used freebsd git mirror on github: https://github.com/freebsd/freebsd-head.

No comments:

Post a Comment