queries, caching of queries, transactions and calling stored procedures. The framework maps JavaBeans to. SQL statements using a XML descriptor. The performance comparison of Hibernate and. iBATIS are explored in this paper. Both of the above tools h
The stationary temporal and (perhaps less convincingly) spatial persistence expo- .... Below Ïc the data in. Fig. 2(a) show no real scaling regime, in analogy to the curvature visible in Ctemp. Â±. (Ï) shown Fig. 2(b). We find no difference between
The coach's task is to provide real-time feedbacks to ensure ... Fitbit Coach provides workout guides .... Sequences of Real World Scenes, IEEE Transactions.
Mar 7, 2016 - coefficients in K and with standard grading, i.e. Sd is the K-vector space of homogeneous polynomials, or forms, of degree d. Given F â Sd, we ...
Jul 30, 2016 - features that were intercorrelated (Pearson's r) greater than 0.95. (absolute .... to the bottom. The Eyelink 1000 (SR Research Ltd., Ottawa, ..... 2519-24, Nov 2007. . P. Tome, R. Vera-Rodriguez, J. Fierrez, and J. Ortega-Garcia,.
Nov 3, 2017 - three fundamental concepts in signal detection theory: the true positive fraction (TPF), false positive fraction (FPF), and detection threshold. By supplying a theoretical basis and recipe for generating the performance map, we hope to
Feb 18, 2015 - Tel.: +386 1 476 81 86. Email addresses: [email protected] (Neli Blagus), [email protected] (Lovro ... benefits, analyzing the changes undergone by networks under the effects of the simplifi- ......  J. Leskovec, L
Anthropology, Schools of Life Sciences and Pharmacy, Fudan University, Shanghai 201203, China. 17 Department of Mathematics and ..... We thank David L. Duewer for review of the manuscript, Cecelie Boysen for discussion of results, and all .... s.nist
determining quality in HE (the research assessment exercise). Metrics2, rather than peer review will be ... A scientist has index h if h of his Np papers have at least h citations each, and the other (Np - h) papers have at most ... published papers
Nov 29, 2016 - In this paper, we present a framework for assessing the effect of non-stationary Gaus- sian noise and radio frequency interference (RFI) on the signal to noise ratio, the number of false positives detected per true positive and the sen
2 Department of Electrical Engineering, COMSATS Institute of Information Technology, Islamabad, Pakistan ... Performance Analysis ... vision systems today. In this work, we propose a generic framework for finding the operating and guarantee regions o
Apr 19, 2016 - this idea to obtain bounds for any phase-insensitive Gaussian bosonic channel. ... arXiv:1511.08710v2 [quant-ph] 19 Apr 2016 ...... [Online]. Available: http://arxiv.org/abs/1510.08863.  R. GarcÄ±a-PatrÃ³n, S. Pirandola, S. Lloyd,
Jun 9, 2017 - for the Newsvendor problems. introducing the Deep Learning neural network architecture and derive the BP algorithm when the proposed L1 loss function is ..... 'randi(20,3)' function, indicating that drawing data from uniform distributio
Jul 29, 2014 - Question-&-Answer (QA) websites have emerged as efficient platforms for knowledge sharing and problem solving. In particular, the Stack Exchange platform includes some of the most popular QA communities to date, such as Stack Over- flo
Mar 11, 2015 - 700 km depending on solar activity. The migrating and semi-diurnal tides are specified at the lower boundary using the Global Scale Wave ...... with the Space Weather Modeling Framework, Space Weather,. 5, 06003, doi:10.1029/2006SW0002
Sep 24, 2017 - Socles and cogenerator functors. 53. 6.5. Closed socles along faces of positive dimension. 54. 6.6. Socles along faces of positive dimension. 58. 7. Essential .... key purposes, as structural supports as well as conduits for airways, n
Oct 28, 2016 - over the LDA and GGA functionals in describing the electronic structure and phonon frequencies, the static structural properties of ... calculations have proved the method of choice for many researches owing to its speed and ... proper
Jan 26, 2011 - In defining the skew product, the contour of integration will be defined such that if t ..... H0du and approaches â in the sector Ï/3 < arg T
Mar 3, 2016 - DAVID PASK, ADAM SIERAKOWSKI, AND AIDAN SIMS. Abstract. We study dimension theory for the Câ-algebras of row-finite k-graphs with.
Feb 1, 2016 - respect to rank and border rank up to SL(3)-action and we examine the differences with the complex ... Statistic, in Computer Science, in Data Analysis and in other scientific areas. This topic is a particular ... The definition of real
Nov 30, 2016 - Abstract. In the data deluge context, pattern recognition or labeling in streams is becoming quite an essential and pressing task as data flows inside always bigger streams. The assessment of such tasks is not so easy when dealing with
Sep 11, 2008 - Bios. 156, 21 (1999).  J. C. Claussen and A. Traulsen, Phys. Rev. Lett. 100,. 058104 (pages 4) (2008).  F. van den Bosch, J. Zadoks, and J. Metz, Phytopathol- ogy 78, 54 (1988).  F. van den Bosch, J. A. J. Metz, and J. C. Z
Sep 11, 2008 - tems are to some degree spatially extended; however, the classical theory of the dynamics of epidemics ..... 1: (Color online) Three types of spatial pattern absent of mobility with ÏI = 0.40, Î²min = 0.60 and Î²max ... using Linux, b
framework, utilizing suitable metrics that mirror real-world performance of different components of the system, is ... proposes a generic framework for finding the operating and guarantee regions of a local feature detector under some ..... These det
Rank Persistence: Assessing the Temporal Performance of Real-World Person Re-Identification
arXiv:1706.00553v2 [cs.CV] 5 Jun 2017
Srikrishna Karanam, Eric Lam, and Richard J. Radke Department of Electrical, Computer, and Systems Engineering Rensselaer Polytechnic Institute Troy, NY, United States [email protected], [email protected]
supervised fashion, a distance metric that is used to search for the person of interest in the gallery set (e.g., [18, 14, 19, 9]).
Designing useful person re-identification systems for real-world applications requires attention to operational aspects not typically considered in academic research. Here, we focus on the temporal aspect of re-identification; that is, instead of finding a match to a probe person of interest in a fixed candidate gallery, we consider the more realistic scenario in which the gallery is continuously populated by new candidates over a long time period. A key question of interest for an operator of such a system is: how long is a correct match to a probe likely to remain in a rank-k shortlist of possible candidates? We propose to distill this information into a Rank Persistence Curve (RPC), which allows different algorithms’ temporal performance characteristics to be directly compared. We present examples to illustrate the RPC using a new long-term dataset with multiple candidate reappearances, and discuss considerations for future re-identification research that explicitly involves temporal aspects.
• Multi-shot re-id, in which both the probe and gallery candidates are represented as short image sequences instead of single frames (e.g., [10, 7, 8, 17]). While critical to the success of a real-world deployment of a re-identification algorithm, research in these areas generally over-simplifies the problem that would face a realworld user of a re-id system. In particular, the temporal aspect of the re-id problem is totally ignored in most academic re-id research. That is, in the real world, candidates would be constantly added to the gallery as new subjects are automatically tracked, as opposed to presented to an algorithm all at once. Even if a correct match to the probe appears in a rank-ordered shortlist shortly after they appear in a gallery camera, this isn’t helpful to a user if the candidate is immediately “pushed off” the list after a few minutes by a new wave of incoming candidates. To the user, a natural question is, how long can a correct match be expected to stay in the shortlist under typical circumstances? Proposed re-id algorithms should be judged based on this notion of persistence in time, not just raw batch performance as presented in a cumulative match characteristic (CMC) curve. Figure 1 illustrates the problem; an incoming correct match arrives at a low rank upon its first appearance, but is quickly pushed down the list as more candidates arrive. In this example, the correct candidate would have only stayed on a shortlist of the top 10 candidates for 10 minutes (about the same duration as a typical coffee break!). In this paper, we explore several temporal aspects of reid and propose new evaluation methodologies that allow different re-id algorithms to be compared based on the concept of rank persistence. We discuss strategies for evaluating algorithms in circumstances when the same person of interest can appear multiple times in the gallery (which could occur
1. Introduction Research in the area of automatic human reidentification, or re-id, has exploded in the past ten years. The re-id problem is generally stated as: given an image of a person of interest as seen in a “probe” camera view, how can we find the same person among a set of candidate people seen in a “gallery” camera view? Re-id research to date typically falls into one or more of the following categories: • Appearance modeling, in which the goal is to design or learn a feature representation for re-id candidates that is invariant to challenges like viewpoint and illumination variation (e.g., [1, 11, 13, 15]) • Metric learning, in which the goal is to learn, in a 1
Figure 1. An example showing the temporal evolution of a gallery set captured in a real surveillance system, and the corresponding timevarying rank for a given query achieved by a re-id algorithm. The candidate arrives at t = 2 at rank 5, and drops in rank over time as more candidates arrive in the target camera.
if the gallery spans several hours or days) as well as when the performance on multiple persons of interest should be aggregated. The key concept we propose is called the Rank Persistence Curve (RPC), which quantifies the percentage of candidates that remain at a certain rank for a given duration. RPCs for different algorithms can be directly compared on the same dataset to allow a user to make informed choices about expected performance in real-world deployments. We illustrate our proposed methodology with examples and experiments drawn from a dataset we collected over many hours at a light rail station, which contains multiple time-stamped reappearances of multiple actors. We conclude the paper with further discussion about the temporal implications of re-id, and suggestions for future research in this area.
2. Preliminaries Although the focus of this paper is not the proposal of a new re-id dataset or algorithm, we need instances of each to illustrate our proposed methodology. In particular, since existing re-id benchmarking datasets lack temporal annotation, we collected a new dataset more suitable for this study. The dataset is composed of 10 hours of surveillance video data from a wall-mounted camera located in an indoor light rail station in the United States. We had 7 known actors participate in the data collection activity. Each ac-
tor appeared once in a morning session, and subsequently re-appeared in the same camera view in multiple afternoon and evening sessions. In each re-appearance, the actor wore slightly different clothes or accessories than in the previous session, to make the re-id problem more realistic and challenging. We used each actor’s appearance in the morning session as the “probe” for subsequent re-id queries. In total, we ground-truthed 3 known re-appearances of each actor, each randomly spread across the 10-hour duration. To generate candidate tracks for the gallery (i.e., for both the actors and the distractor pedestrians that entered the camera view over the 10-hour video), we used an off-theshelf person detector, based on the aggregated channel features (ACF) algorithm of Doll´ar et al. [4, 3], to crop out person images. Sample images from the different appearances of the actors in our dataset are shown in Figure 2. A statistical summary of our dataset is provided in Table 1. Table 1. Statistical summary of our dataset
Property # Bounding Boxes # Known Actors # Known Actor Re-Appearances # Candidates (incl. actors) # Video Hours Detection algorithm
Total/Type 4,639 7 3 535 10 ACF [4, 3]
As a baseline re-id algorithm, we used the recently pro-
Figure 2. Examples of actors and their appearances in our dataset.
posed Gaussian of Gaussian (GOG) descriptor  to rank gallery candidates. GOG is an unsupervised algorithm that constructs an appearance model given image data and uses Euclidean distance to rank candidates. The baseline algorithm is multi-shot in that it uses multiple frames of each candidate’s appearance to form feature vectors to be compared, instead of a single rectangle. Specifically, given a track of images for the probe appearance of the person of interest and each gallery candidate, we extract features for each image using GOG. Let xkp , k = 1, · · · , n and xki , k = 1, · · · , m denote the n feature vectors of the probe and the m feature vectors of the ith gallery candidate, respectively. We then determine the appearance model for the probe, fp , and the gallery candidate, fi , as the mean feature vector of the available feature vectors. Specifically, n
1X k xp n k=1 m
1 X k xi m k=1
The similarity score si , used to rank gallery candidates, is computed using the Euclidean distance as si = kfp − fi k
3. Rank Persistence In this section, we present the concept of rank persistence in steps, working up to evaluating situations in which multiple persons of interest each appear multiple times over the course of a long video sequence.
3.1. One probe, one reappearance We first consider the case of a single probe/query that has exactly one reappearance over the course of an entire video. In this case, we can easily graph their rank over time with respect to an ever-increasing gallery, as illustrated in Figure 3. The horizontal axis is real time (i.e., minutes from the beginning of our 10-hour test sequence). The vertical axis shows the rank of the true reappearance in the gallery over time. Clearly, the rank can only decrease as more candidates arrive in the gallery. Also, the graph does not start at t = 0 but indicates the time at which the reappearance occurs. These per-probe curves will form the basis for the aggregate Rank Persistence Curve discussed in Section 3.3. Figure 4 illustrates three rank curves for probe/gallery pairs of increasing difficulty. In the first example, the gallery reappearance is very similar to the probe, and also dissimilar to other candidates seen over the course of the video, so the rank is fairly constant at a low value. In the last example, the gallery reappearance is quite unlike the
Figure 3. An example of how the rank r of a person of interest changes in a temporally-varying gallery set. This candidate’s first re-appearance is at t = 2 minutes.
probe, so the rank increases quickly as new candidates appear.
3.2. One probe, multiple reappearances In a real-world scenario, when we are searching for a person of interest in streaming video over many hours or even days, it is possible that s/he may re-appear multiple times at different ranks. Therefore, we modify the temporal rank curve from the previous section as illustrated in Figure 5. That is, the vertical axis shows the highest instantaneous rank held by any reappearance of the probe. In this example, the first reappearance of the candidate in the gallery looks rather unlike the probe, so the initial rank is relatively high and increases sharply with time. However, in the next reappearance, the gallery image looks very much like the probe; the rank of this new candidate is low, and the candidate persists at this rank for the rest of the long video. A third reappearance of the candidate occurs later in the video, but is not as similar as the second reappearance, so does not affect the temporal rank plot.
3.3. Multiple probes, multiple reappearances Finally, we arrive at the most general situation, in which we characterize the performance of a given re-id algorithm across multiple probes, each of which may have multiple reappearances in the gallery. We define the Rank Persistence Curve (RPC) to evaluate performance as follows. First, we fix a specific rank r. For each duration d (in real units) on the horizontal axis, we plot the percentage of candidates that appear continuously in the top-r list for at least d units. Thus, the RPC is monotonically decreasing, and RPCs at higher ranks dominate those at lower ranks.
Figure 6a illustrates RPCs for our dataset/baseline reid algorithm combination. In contrast, a traditional Cumulative Match Characteristic (CMC) curve, also for our dataset/baseline re-id algorithm combination (averaging all available feature vectors for every candidate), is shown in Figure 6b. We can see that the two types of curves are qualitatively different. Since we want to capture the temporal aspect of rank in the RPC, the dependent axis is no longer rank but duration, and we need a third “axis” (in this case color) to indicate rank. Let’s focus on the RPC for r = 1, shown in red. This captures our objective of visualizing how likely and how long a candidate is to stay at rank 1 across a long video sequence. Clearly, this is a stringent requirement, and we can see that only 1 of the 7 probes ever had a reappearance at rank 1 at all. If we consider the RPC for r = 5, we see that more candidates are likely to persist at rank 5 for longer amounts of time. RPCs can help operators of re-id systems answer questions like, can my re-id algorithm be expected to preserve correct matches in the top-10 shortlist for at least 15 minutes at least 90% of the time? These considerations are important both in terms of the length of the shortlist (real-world end users, typically not computer vision experts, would not want to scroll through pages and pages of candidates to find the person of interest) and the duration of persistence (in real-world scenarios, end users may only get around to checking the output of a re-id surveillance system a few times an hour). Thus, RPCs can be used to quickly visualize the performance of competing re-id systems applied in the same environment. For example, Figure 7 shows the RPCs for our baseline algorithm using GOG vs. the ColorTexture algorithm proposed by Gray and Tao , which uses an ensemble of localized color and texture features. We can see that the GOG algorithm dominates the older algorithm with respect to temporal persistence, and that the RPCs capture the essence of both algorithms’ performance in an easy-to-read graph.
3.4. Video Flow Density We conclude this section with a brief discussion on video flow density, a topic we have not considered in this paper but has a direct impact on the behavior of the rank of the person of interest. We define video flow density as the instantaneous number of people seen by the gallery camera per unit time. To understand how this might impact rank persistence, consider two nonoverlapping time blocks with the same duration. In the first block, the gallery camera sees x people walking by. In the second block, the gallery camera sees y people walking by. If x y, we add many more people in the gallery in the first time block, likely resulting in a steeper increase in a candidate’s re-id rank compared to the second time block. This suggests we should pay close
Figure 4. Temporal rank curves for three sample queries. In each case, the image on the top is the probe/query and the image on the bottom is the known reappearance in the gallery. The graph to the right of these images shows the temporal evolution of the rank of the known reappearance.
Figure 5. Temporal rank plot for a query with multiple re-appearances.
attention to the rate at which new people in the video are added to the gallery.
4. Discussion and Future Work Now that the underlying computer vision and machine learning technologies for re-id have matured [6, 21], we contend that researchers should begin to take a broader view of evaluating how re-id algorithms should integrate
into functional real-world systems. For example, as noted in Camps et al. , the problem of comparing one candidate rectangle of pixels to another is only a small part of a fully automated re-id system. Instead, we must take into account that the candidate rectangles are likely generated by an automatic (and possibly inaccurate) human detection and tracking subsystem, that the overall system needs to operate in real time, and that the system may be in operation for very long periods of time. Instead of benchmarking datasets in
Rank Persistence Curve
Matching Rate (%)
Cumulative Match Characteristic Curve
r=50 r=20 r=10 r=5 r=1
50 40 30
70 60 50 40 30
Figure 6. (a) The proposed Rank Persistence Curve (RPC) at different ranks for our dataset/baseline re-id algorithm combination. (b) The traditional Cumulative Match Characteristic (CMC) curve for our dataset/baseline re-id algorithm combination.
Figure 7. RPCs can be used to easily compare the temporal performance of competing re-id algorithms. In this example, the GOG algorithm’s RPC (solid curve) dominates the ColorTexture’s algorithm’s RPC (dashed curve) at r = 5 and r = 20, demonstrating the superiority of GOG from a temporal persistence perspective.
which the gallery images are acquired only a few moments after the probe images, we must consider crime prevention applications in which a perpetrator may return to the scene
of the crime days after their initial detection. In such cases, the gallery of candidates is ever-expanding, and for long periods of time may not contain the person of interest at all. These real-world considerations present several challenges to the re-id research community, several of which we discuss below. • While they are derived from real-world surveillance video, current re-id benchmarking datasets such as VIPeR , iLIDS-VID , and MARS  lack time stamps for the gallery sets, rendering them inappropriate for the type of temporal research discussed here. While it might be possible to repurpose them for temporal re-id research, adding artificial time stamps seems suboptimal. It would be better to generate a new temporal re-id dataset for this purpose, ideally spanning several days and thousands of tracked candidates. For validation, multiple reappearances of a substantial number of actors would need to be included. The dataset discussed in this paper is a step in this direction, but has too few unique actors to be broadly useful. • The concept of “splits” is critical for fair comparisons of re-id algorithms using current benchmarking datasets. That is, the provider of the dataset typically specifies a random subset of the data to be used for training and another to be used for testing. A similar concept would need to be developed for temporal reid datasets, i.e., not only specifying which candidates
should be used for training/testing, but also which temporal spans should be used in the time-evolving galleries. 
• As discussed in Section 3.4, the x-axis of the proposed Rank Persistence Curves is critically coupled to the video flow density of candidates in the underlying camera. Operationally, it would be more useful to characterize rank persistence in real units of minutes or hours; however, this ties the RPC strongly to a particular dataset and possibly a time of day. For example, different rail stations may have different levels of traffic, and the traffic would vary during rush hour vs. off-peak times. Temporal re-id benchmarking protocols would need to distinguish subsets of data with respect to video flow rate. • In our experience with integrating academic re-id algorithms into operational surveillance command centers, we found the issue of user interfaces to be extremely important. The similarity between a rank-k shortlist and a police lineup was an effective analogy. However, the potentially very long time scales for crime prevention applications requires the re-evaluation of an operationally meaningful shortlist. Should candidates “age out” of the ranked list using some sort of forgetting factor? Should extremely promising candidates from long ago be kept alongside less-certain but more timely recent candidates? Should the time-varying gallery contain all candidates ever seen or only those from the last N minutes? These considerations require close consultation with the potential users of the system to understand and set expectations and corresponding interface choices.
Acknowledgements This material is based upon work supported by the U.S. Department of Homeland Security, Science and Technology Directorate, Office of University Programs, under Grant Award 2013-ST-061-ED0001. The views and conclusions contained in this document are those of the authors and should not be interpreted as necessarily representing the official policies, either expressed or implied, of the U.S. Department of Homeland Security. Thanks to John Joyce for supplying the video data.
References  L. Bazzani, M. Cristani, and V. Murino. Symmetry-driven accumulation of local features for human characterization and re-identification. Comput. Vision and Image Understanding (CVIU), 117(2):130–144, 2013. 1  O. Camps, M. Gou, T. Hebble, S. Karanam, O. Lehmann, Y. Li, R. Radke, Z. Wu, and F. Xiong. From the lab to the real
world: Re-identification in an airport camera network. IEEE Transactions on Circuits and Systems for Video Technology (T-CSVT), 27(3):540–553, 2017. 5 P. Doll´ar, R. Appel, S. Belongie, and P. Perona. Fast feature pyramids for object detection. IEEE Trans. Pattern Anal. Mach. Intell. (T-PAMI), 36(8):1532–1545, 2014. 2 P. Doll´ar, S. J. Belongie, and P. Perona. The fastest pedestrian detector in the west. In Proc. Brit. Mach. Vision Conf. (BMVC), 2010. 2 D. Gray and H. Tao. Viewpoint invariant pedestrian recognition with an ensemble of localized features. In Eur. Conf. Comput. Vision (ECCV). 2008. 4, 6 S. Karanam, M. Gou, Z. Wu, A. Rates-Borras, O. I. Camps, and R. J. Radke. A comprehensive evaluation and benchmark for person re-identification: Features, metrics, and datasets. arXiv preprint arXiv:1605.09653, 2016. 5 S. Karanam, Y. Li, and R. J. Radke. Person re-identification with block sparse recovery. Image and Vision Computing, 60:75–90, 2017. 1 Y. Li, Z. Wu, S. Karanam, and R. J. Radke. Multi-shot human re-identification using adaptive fisher discriminant analysis. In Proc. Brit. Mach. Vision Conf. (BMVC), 2015. 1 S. Liao, Y. Hu, X. Zhu, and S. Z. Li. Person re-identification by local maximal occurrence representation and metric learning. In IEEE Conf. Comput. Vision and Pattern Recognition (CVPR), 2015. 1 G. Lisanti, I. Masi, A. D. Bagdanov, and A. Del Bimbo. Person re-identification by iterative re-weighted sparse ranking. IEEE Trans. Pattern Anal. Mach. Intell. (T-PAMI), 37(8):1629–1642, 2015. 1 B. Ma, Y. Su, and F. Jurie. Local descriptors encoded by Fisher vectors for person re-identification. In ECCV Workshops, 2012. 1 T. Matsukawa, T. Okabe, E. Suzuki, and Y. Sato. Hierarchical Gaussian descriptor for person re-identification. In IEEE Conf. Comput. Vision and Pattern Recognition (CVPR), 2016. 3 N. McLaughlin, J. Martinez del Rincon, and P. Miller. Recurrent convolutional network for video-based person reidentification. In IEEE Conf. Comput. Vision and Pattern Recognition (CVPR), 2016. 1 B. Prosser, W.-S. Zheng, S. Gong, and T. Xiang. Person reidentification by support vector ranking. In Proc. Brit. Mach. Vision Conf. (BMVC), 2010. 1 R. Satta. Appearance descriptors for person re-identification: a comprehensive review. arXiv preprint arXiv:1307.5748, 2013. 1 T. Wang, S. Gong, X. Zhu, and S. Wang. Person reidentification by video ranking. In Eur. Conf. Comput. Vision (ECCV). 2014. 6 T. Wang, S. Gong, X. Zhu, and S. Wang. Person reidentification by discriminative selection in video ranking. IEEE Trans. Pattern Anal. Mach. Intell. (T-PAMI), 38(12):2501–2514, 2016. 1 K. Q. Weinberger and L. K. Saul. Distance metric learning for large margin nearest neighbor classification. 10:207–244, 2009. 1
 F. Xiong, M. Gou, O. Camps, and M. Sznaier. Person reidentification using kernel-based metric learning methods. In Eur. Conf. Comput. Vision (ECCV). 2014. 1  L. Zheng, Z. Bie, Y. Sun, J. Wang, C. Su, S. Wang, and Q. Tian. MARS: A video benchmark for large-scale person re-identification. In Eur. Conf. Comput. Vision (ECCV), 2016. 6  L. Zheng, Y. Yang, and A. G. Hauptmann. Person reidentification: Past, present and future. arXiv preprint arXiv:1610.02984, 2016. 5