Coding: Fleeting Thoughts

A place to discuss the implementation and style of computer programs.

Moderators: phlip, Moderators General, Prelates

EvanED
Posts: 4331
Joined: Mon Aug 07, 2006 6:28 am UTC
Location: Madison, WI
Contact:

Re: Coding: Fleeting Thoughts

Postby EvanED » Sun Sep 08, 2013 12:23 am UTC

Here's another interesting aspect of big-O analysis. I'm not saying that it's particularly relevant for this conversation, but it's something that you should always keep in mind when dealing with asymptotic complexity.

Take the following example, which is from an article by Bjarne Stroustrup that I learned about from Yakk a while back on these forums. Suppose I have a C++ vector and a C++ list. To each, I perform the following operation. First, I generate a bunch of random numbers, finding the place in the sequence that it should go in sorted order and inserting the number there. This has to be a linear scan for the list, so to make it fair I do a linear scan of the vector as well rather than a binary search. In the same order I generated the numbers, I then go and remove them from the list, again doing a linear scan to find the appropriate place and then removing.

In both cases, the "find the right place" is linear in the current size of the sequence. For the vector, the "insert/delete" operation is linear in the size of the sequence. For the list, the insert/delete operation is constant.

So, the question is: at around what number of elements do you expect that the additional time spent for the linear insert/delete in the vector will cause the vector to fall behind in performance relative to the list? Answer in the spoiler.

Spoiler:
Actually: never.

Said another way: replacing the O(n) insert/replace operation with a O(1) insert/replace operation will never pay off.

Why is this? How can it be? And the answer is that looking at just the insert/replace operation is the wrong granularity, because that's not what was being timed. The entire operation was being timed, and that is the same complexity (O(n2)) in both cases.

And when we replaced the vector with the list, the decrease of time spent doing insert/delete was more than offset by a linear increase in time elsewhere.


(I should also say that I carried out this test myself, using a few different configurations. My results agree with Stroustrup's, and seem to be pretty robust to changes in the exact process, including changes that cause all nodes in the list to be allocated in sorted order.)

User avatar
You, sir, name?
Posts: 6983
Joined: Sun Apr 22, 2007 10:07 am UTC
Location: Chako Paul City
Contact:

Re: Coding: Fleeting Thoughts

Postby You, sir, name? » Sun Sep 08, 2013 12:36 am UTC

Diadem wrote:
You, sir, name? wrote:I disagree. Big-O only gives you an upper bound for how an algorithm scales for large data sets, it provides no information about performance at a given value n. It's often a decent indicator, but a programmer should be wary about reading too much into it. After all, the only true metric of computational performance is processing time.

Theoretically, that is all true. However in practice the lower-complexity algorithm is almost always faster. Exceptions are very rare indeed. I mean sure, you should be aware of them, I agree. But let's not pretend this happens every day.

There may certainly be factors that make O(n2) faster than O(n log n) -- perhaps the former is cache-aware and gets nicely vectorized, whereas the latter stumbles through the memory like a bull in a china shop and fails vectorization completely.

Well for large enough n (and not using special cases, like using already nearly sorted arrays as input for your sorting algorithm) your O(n log n) will always be faster than your O(n^2). That's pretty much the definition of big O notation. Caching and vectorization change nothing about that.


The point is, for large enough values of n isn't the same as any given value n. In the scenario I listed above, for example, n is of order 100, but the function gets called billions of times per hour.
I edit my posts a lot and sometimes the words wrong order words appear in sentences get messed up.

User avatar
Yakk
Poster with most posts but no title.
Posts: 11129
Joined: Sat Jan 27, 2007 7:27 pm UTC
Location: E pur si muove

Re: Coding: Fleeting Thoughts

Postby Yakk » Sun Sep 08, 2013 1:22 am UTC

O(n^2) is *often* faster than a O(n lg n) algorithm, because n lg n algorithms are often much fancier, and often jump around in memory. n^2 algorithms are often simple, and just go over some memory in order a bunch of times.

And n is often small.

The lg n component is so small it may as well be a constant on reasonable n: but what it signifies is a large constant factor from the more complex algorithm usually.

However, n lg n while slow, does not slow down as much in the unusual cases. If you make your software fast enough and keep your algorithms sub quadratic, on massive unexpected data you will not explode.

Amusingly, when I find painful slowdowns in real commercial software on large data loads, it usually is not the merely quadratic, but the cubic or higher. Often some subcomponent is quadratic (and is not the problem) but then it gets called linearly often. Had the subcomponent fought to stay log linear, the linear tines it is called may not have been the major slowdown problem, and some other component would instead get the development attention to speed it up.
One of the painful things about our time is that those who feel certainty are stupid, and those with any imagination and understanding are filled with doubt and indecision - BR

Last edited by JHVH on Fri Oct 23, 4004 BCE 6:17 pm, edited 6 times in total.

User avatar
Jplus
Posts: 1721
Joined: Wed Apr 21, 2010 12:29 pm UTC
Location: Netherlands

Re: Coding: Fleeting Thoughts

Postby Jplus » Sun Sep 08, 2013 7:11 am UTC

EvanED wrote:Take the following example, which is from an article by Bjarne Stroustrup that I learned about from Yakk a while back on these forums. Suppose I have a C++ vector and a C++ list. To each, I perform the following operation. First, I generate a bunch of random numbers, finding the place in the sequence that it should go in sorted order and inserting the number there. This has to be a linear scan for the list, so to make it fair I do a linear scan of the vector as well rather than a binary search. In the same order I generated the numbers, I then go and remove them from the list, again doing a linear scan to find the appropriate place and then removing.

In both cases, the "find the right place" is linear in the current size of the sequence. For the vector, the "insert/delete" operation is linear in the size of the sequence. For the list, the insert/delete operation is constant.

So, the question is: at around what number of elements do you expect that the additional time spent for the linear insert/delete in the vector will cause the vector to fall behind in performance relative to the list? Answer in the spoiler.

Spoiler:
Actually: never.

Said another way: replacing the O(n) insert/replace operation with a O(1) insert/replace operation will never pay off.

Why is this? How can it be? And the answer is that looking at just the insert/replace operation is the wrong granularity, because that's not what was being timed. The entire operation was being timed, and that is the same complexity (O(n2)) in both cases.

And when we replaced the vector with the list, the decrease of time spent doing insert/delete was more than offset by a linear increase in time elsewhere.

I always found this a good example of not explicitly calculating cache complexity. The full test in a list takes O(n) cache misses while the same test in a vector takes only O((n/k)^2) cache misses, with k the size of the cache. Cache misses happen to be a lot more expensive than block moves.

It's not true that complexity changes when you change granularity, though. The same algorithm will have the same asymptotic number of core operations no matter how you look at it. The only difference you can make is by more completely identifying the core operations and by better analyzing how expensive they really are. The linked list version is still O(n) but with a giant constant factor while the vector version is O(n2) with a tiny constant factor. The reason you see a curve in the time-used graph for the linked list is that the number of cache misses is less than asymptotic for small n.
"There are only two hard problems in computer science: cache coherence, naming things, and off-by-one errors." (Phil Karlton and Leon Bambrick)

coding and xkcd combined

(Julian/Julian's)

User avatar
Thesh
Made to Fuck Dinosaurs
Posts: 6598
Joined: Tue Jan 12, 2010 1:55 am UTC
Location: Colorado

Re: Coding: Fleeting Thoughts

Postby Thesh » Sun Sep 08, 2013 7:39 am UTC

Jplus wrote:It's not true that complexity changes when you change granularity, though. The same algorithm will have the same asymptotic number of core operations no matter how you look at it.


It can when you process in fixed-size blocks. Take computing the hash of a password vs a software package, with SHA-384 or SHA-512 you have a block size of 1024 bits. Realistically, hashing a password will take O(1) since it is the same number of operations as long as the password is under 256 ASCII characters. However, the package will take O(n) to compute the hash because packages will likely be 20+ kilobytes.
Summum ius, summa iniuria.

User avatar
Jplus
Posts: 1721
Joined: Wed Apr 21, 2010 12:29 pm UTC
Location: Netherlands

Re: Coding: Fleeting Thoughts

Postby Jplus » Sun Sep 08, 2013 9:30 am UTC

Then the complexity is just O(n). Big-O is only about asymptotic behaviour, not about very small inputs such as single passwords or linked lists that reside entirely inside the cache (note that the cache is also a "fixed-size block"). This is of course also one of the most important limitations of big-O, for many practical situations it's not that informative.
"There are only two hard problems in computer science: cache coherence, naming things, and off-by-one errors." (Phil Karlton and Leon Bambrick)

coding and xkcd combined

(Julian/Julian's)

EvanED
Posts: 4331
Joined: Mon Aug 07, 2006 6:28 am UTC
Location: Madison, WI
Contact:

Re: Coding: Fleeting Thoughts

Postby EvanED » Sun Sep 08, 2013 6:42 pm UTC

Jplus wrote:It's not true that complexity changes when you change granularity, though.
I didn't say it changes, just that looking at the wrong granularity is meaningless. The insertion of a single number in the vector is O(n) + O(n) = O(n) and in the list it's O(n) + O(1) = O(n). What I'm saying is that looking at the bolded operations (the complexity of the insert and remove) is unhelpful because it's the wrong granularity, and you have to look at the overall time. The insert/delete in the list is still O(1) -- that doesn't change when you look at the bigger picture -- but it doesn't matter either because you're doing complexity analysis of something that you're not actually measuring or care about.

The full test in a list takes O(n) cache misses while the same test in a vector takes only O((n/k)^2) cache misses, with k the size of the cache. ... The linked list version is still O(n) but with a giant constant factor while the vector version is O(n2) with a tiny constant factor. The reason you see a curve in the time-used graph for the linked list is that the number of cache misses is less than asymptotic for small n.
Uh, how do you get the O(n) vs O(n2) cache misses?

It seems to me the list will incur a cache miss at each node traversal, which is O(n) misses for each insertion/deletion step when getting to the appropriate place, then none thereafter, so if that's what you're measuring I agree there. (That step is carried out n times, so that's O(n2) cache misses total, which is why that statement is qualified.)

Meanwhile, for the vector, during the traversal step there will be a cache miss every k elements, which is still O(n) (with a fixed k). (Actually you could argue that prefetchers would make it O(1). Dunno if this holds on real hardware, but I can imagine it could.) Then for the insertion step, it starts at the back and shifts elements forward by one slot. (I'm assuming no reallocation, but that doesn't introduce an n2 anywhere I see either.) Again, it should only make a cache miss every k items, for O(n)[//i] misses. (And again, a prefetecher might even make this [i]O(1) as well, though I don't know if they see linear access backwards patterns.) So I only see O(n) misses (at most!) with the vector version each insert/delete, or O(n2) total. Potentially that could be O(1) misses each insert/delete and O(n) total!

What about emperically?

Well, here's the graph that one guy did:
Image

And Stroustrup's results:
Image

I don't think either carried out formal complexity analysis to determine what kind of curve better fits, but in both cases if exactly one of the list/vector has a super-linear number of cache misses per insert/delete, "the vector" wouldn't be my choice for which one it is.

User avatar
Jplus
Posts: 1721
Joined: Wed Apr 21, 2010 12:29 pm UTC
Location: Netherlands

Re: Coding: Fleeting Thoughts

Postby Jplus » Sun Sep 08, 2013 8:13 pm UTC

EvanED wrote:
Jplus wrote:It's not true that complexity changes when you change granularity, though.
I didn't say it changes, just that looking at the wrong granularity is meaningless. The insertion of a single number in the vector is O(n) + O(n) = O(n) and in the list it's O(n) + O(1) = O(n). What I'm saying is that looking at the bolded operations (the complexity of the insert and remove) is unhelpful because it's the wrong granularity, and you have to look at the overall time. The insert/delete in the list is still O(1) -- that doesn't change when you look at the bigger picture -- but it doesn't matter either because you're doing complexity analysis of something that you're not actually measuring or care about.

Well, I happened to have an a priori interpretation of "granularity" that seems to be incompatible with yours. Sorry for the confusion.

EvanED wrote:
The full test in a list takes O(n) cache misses while the same test in a vector takes only O((n/k)^2) cache misses, with k the size of the cache. ...
Uh, how do you get the O(n) vs O(n2) cache misses?

It seems to me the list will incur a cache miss at each node traversal, which is O(n) misses for each insertion/deletion step when getting to the appropriate place, then none thereafter, so if that's what you're measuring I agree there.

No, that's not what I was measuring. To my embarassment I now realise I forgot about the node traversals. :oops:
So yes, either way you're completely right that both are O(n2).

The rest of what you said in your post was already undisputed as far as I'm concerned (as in, I already agreed with that).
"There are only two hard problems in computer science: cache coherence, naming things, and off-by-one errors." (Phil Karlton and Leon Bambrick)

coding and xkcd combined

(Julian/Julian's)

lgw
Posts: 437
Joined: Mon Apr 12, 2010 10:52 pm UTC

Re: Coding: Fleeting Thoughts

Postby lgw » Mon Sep 09, 2013 7:46 pm UTC

I sometimes have to remind young software engineers that "code does not run on a whiteboard", and thus considerations of asymptotic performance rarely matter. The only thing that matters to the performance of a program is wherever the bottleneck is (CPU, disk, network, or locking). There are very few kinds of programs where CPU will be a bottleneck, and it's rare that there's a way to improve the asymptotic performance if I/O, so usually performance improvement is about finding the design flaw that's causing too much lock contention. Often the design flaw came from someone trying to optimize CPU on a system that never got above 5% CPU to begin with. :roll:

But, to agree with others here, if CPU ever does need optimizing, locality of reference (making the best use of in-CPU cache) is usually the important thing.
"In no set of physics laws do you get two cats." - doogly

User avatar
Yakk
Poster with most posts but no title.
Posts: 11129
Joined: Sat Jan 27, 2007 7:27 pm UTC
Location: E pur si muove

Re: Coding: Fleeting Thoughts

Postby Yakk » Mon Sep 09, 2013 11:48 pm UTC

Yes, but you should write code with sane O notation performance to start with. Writing that code is easier than optimizing. Then optimize after you find slow code. You can trade programmer time for performance, and you might as well do it where it matters. With good big O, your code at least won't be insanely slow when your typical cases (which get tested more) scale up a few 100 or 1000 times.
One of the painful things about our time is that those who feel certainty are stupid, and those with any imagination and understanding are filled with doubt and indecision - BR

Last edited by JHVH on Fri Oct 23, 4004 BCE 6:17 pm, edited 6 times in total.

User avatar
Xeio
Friends, Faidites, Countrymen
Posts: 5101
Joined: Wed Jul 25, 2007 11:12 am UTC
Location: C:\Users\Xeio\
Contact:

Re: Coding: Fleeting Thoughts

Postby Xeio » Tue Sep 10, 2013 3:50 am UTC

Wow. I somehow did something so horrible with Find Replace (just on a copyright header! It's a comment!) that Visual Studio now refuses to open several of my code files (or compile them)...

It opens up notepad instead... and if I try to open the files using the compile errors, I get an error that VS doesn't know how to open .cs files...

WHAT THE.

User avatar
phlip
Restorer of Worlds
Posts: 7573
Joined: Sat Sep 23, 2006 3:56 am UTC
Location: Australia
Contact:

Re: Coding: Fleeting Thoughts

Postby phlip » Tue Sep 10, 2013 4:21 am UTC

Have you tried turning it off and turning it back on again?

Code: Select all

enum ಠ_ಠ {°□°╰=1, °Д°╰, ಠ益ಠ╰};
void ┻━┻︵​╰(ಠ_ಠ ⚠) {exit((int)⚠);}
[he/him/his]

User avatar
Xeio
Friends, Faidites, Countrymen
Posts: 5101
Joined: Wed Jul 25, 2007 11:12 am UTC
Location: C:\Users\Xeio\
Contact:

Re: Coding: Fleeting Thoughts

Postby Xeio » Tue Sep 10, 2013 4:27 am UTC

Deleting some whitespace seems to fix it (from notepad). Supposedly I managed to inject EoF terminators in somehow if the compile errors were to be believed...

User avatar
Yakk
Poster with most posts but no title.
Posts: 11129
Joined: Sat Jan 27, 2007 7:27 pm UTC
Location: E pur si muove

Re: Coding: Fleeting Thoughts

Postby Yakk » Tue Sep 10, 2013 5:34 am UTC

3 favorite VS issues:

Last line ending with a comment. (really annoying one)

Failure of last line to contain a newline. (fails to be a proper text file)

CR/LF vs CR inconsistencies. (I believe I've had issues)
One of the painful things about our time is that those who feel certainty are stupid, and those with any imagination and understanding are filled with doubt and indecision - BR

Last edited by JHVH on Fri Oct 23, 4004 BCE 6:17 pm, edited 6 times in total.

User avatar
You, sir, name?
Posts: 6983
Joined: Sun Apr 22, 2007 10:07 am UTC
Location: Chako Paul City
Contact:

Re: Coding: Fleeting Thoughts

Postby You, sir, name? » Tue Sep 10, 2013 6:48 am UTC

I wish I had VS problems.

Yesterday, I tried to run a JUit test in eclipse. This triggered not the unit test to run, but the Problems tab to jerk back and forth 5 pixels on the screen for a while. And then the screen went gray. And then it went back to normal, but had an abandoned XWindow 10x10 pixels stuck in the middle of it.

Never ran that unit test. Just killed eclipse, checked in the damn code, went home, and let the continuous integration server run it.
I edit my posts a lot and sometimes the words wrong order words appear in sentences get messed up.

User avatar
Diadem
Posts: 5654
Joined: Wed Jun 11, 2008 11:03 am UTC
Location: The Netherlands

Re: Coding: Fleeting Thoughts

Postby Diadem » Tue Sep 10, 2013 7:19 am UTC

You, sir, name? wrote:Never ran that unit test. Just killed eclipse, checked in the damn code, went home, and let the continuous integration server run it.

That's the spirit. Unit tests are for wimps. Real programmers know when code they've written is correct, they don't need no unit tests to confirm what they know in their guts.

Bonus points are awarded for checking in untested code on a Friday, just before going on a 2-week vacation.
It's one of those irregular verbs, isn't it? I have an independent mind, you are an eccentric, he is round the twist
- Bernard Woolley in Yes, Prime Minister

User avatar
Thesh
Made to Fuck Dinosaurs
Posts: 6598
Joined: Tue Jan 12, 2010 1:55 am UTC
Location: Colorado

Re: Coding: Fleeting Thoughts

Postby Thesh » Tue Sep 10, 2013 7:22 am UTC

What is this "testing" you speak of?
Summum ius, summa iniuria.

User avatar
skeptical scientist
closed-minded spiritualist
Posts: 6142
Joined: Tue Nov 28, 2006 6:09 am UTC
Location: San Francisco

Re: Coding: Fleeting Thoughts

Postby skeptical scientist » Tue Sep 10, 2013 7:26 am UTC

You, sir, name? wrote:I wish I had VS problems.

Yesterday, I tried to run a JUit test in eclipse. This triggered not the unit test to run, but the Problems tab to jerk back and forth 5 pixels on the screen for a while. And then the screen went gray. And then it went back to normal, but had an abandoned XWindow 10x10 pixels stuck in the middle of it.

Never ran that unit test. Just killed eclipse, checked in the damn code, went home, and let the continuous integration server run it.

It could be worse. You could be using XCode.
I'm looking forward to the day when the SNES emulator on my computer works by emulating the elementary particles in an actual, physical box with Nintendo stamped on the side.

"With math, all things are possible." —Rebecca Watson

User avatar
Xenomortis
Not actually a special flower.
Posts: 1456
Joined: Thu Oct 11, 2012 8:47 am UTC

Re: Coding: Fleeting Thoughts

Postby Xenomortis » Tue Sep 10, 2013 8:30 am UTC

Diadem wrote:Unit tests are for wimps.

I've never seen a unit test at my current job.

Diadem wrote:Bonus points are awarded for checking in untested code on a Friday, just before going on a 2-week vacation.

But it worked on my computer!
Image

User avatar
TheChewanater
Posts: 1279
Joined: Sat Aug 08, 2009 5:24 am UTC
Location: lol why am I still wearing a Santa suit?

Re: Coding: Fleeting Thoughts

Postby TheChewanater » Tue Sep 10, 2013 12:36 pm UTC

Diadem wrote:
You, sir, name? wrote:Never ran that unit test. Just killed eclipse, checked in the damn code, went home, and let the continuous integration server run it.

That's the spirit. Unit tests are for wimps. Real programmers know when code they've written is correct, they don't need no unit tests to confirm what they know in their guts.

Bonus points are awarded for checking in untested code on a Friday, just before going on a 2-week vacation.



╔═════════════════ ೋღ☃ღೋ ════════════════╗
~ ~ ~ ~ ~ ~ ~ ~ ~ Repost this if ~ ~ ~ ~ ~ ~ ~ ~ ~
~ ~ ~ ~ you are a beautiful strong programmer ~ ~ ~
~ ~ ~ ~ ~ ~ ~ who don’t need no unit tests ~ ~ ~ ~ ~ ~ ~
╚═════════════════ ೋღ☃ღೋ ════════════════╝
ImageImage
http://internetometer.com/give/4279
No one can agree how to count how many types of people there are. You could ask two people and get 10 different answers.

User avatar
skeptical scientist
closed-minded spiritualist
Posts: 6142
Joined: Tue Nov 28, 2006 6:09 am UTC
Location: San Francisco

Re: Coding: Fleeting Thoughts

Postby skeptical scientist » Tue Sep 10, 2013 2:05 pm UTC

Diadem wrote:Unit tests are for wimps.

Thesh wrote:What is this "testing" you speak of?

Xenomortis wrote:I've never seen a unit test at my current job.

TheChewanater wrote:a beautiful strong programmer who don’t need no unit tests

Y'all are fucking psychos.
I'm looking forward to the day when the SNES emulator on my computer works by emulating the elementary particles in an actual, physical box with Nintendo stamped on the side.

"With math, all things are possible." —Rebecca Watson

User avatar
Xenomortis
Not actually a special flower.
Posts: 1456
Joined: Thu Oct 11, 2012 8:47 am UTC

Re: Coding: Fleeting Thoughts

Postby Xenomortis » Tue Sep 10, 2013 2:14 pm UTC

My post was not a jest.

I have, in total, something like 18 months programming experience (I'm 22), 15 months of which have been at this job.
If I have psychopathic tendencies, they came from a neglectful and abusive upbringing.

(¬_¬)
Image

User avatar
You, sir, name?
Posts: 6983
Joined: Sun Apr 22, 2007 10:07 am UTC
Location: Chako Paul City
Contact:

Re: Coding: Fleeting Thoughts

Postby You, sir, name? » Tue Sep 10, 2013 4:27 pm UTC

My previous employer had no testing at all. Each version got to run in a staging environment (where at best it got some monkey testing) before being deployed onto production. If shit hit the fan, you fixed it *in production*.

But my current developer is all about unit tests, and formal testing (we have people with the job title "tester", that test stuff!), continuous integration, and stuff like that. Heck, they even put Clean Code on the required reading list.
Last edited by You, sir, name? on Tue Sep 10, 2013 4:33 pm UTC, edited 1 time in total.
I edit my posts a lot and sometimes the words wrong order words appear in sentences get messed up.

User avatar
Jplus
Posts: 1721
Joined: Wed Apr 21, 2010 12:29 pm UTC
Location: Netherlands

Re: Coding: Fleeting Thoughts

Postby Jplus » Tue Sep 10, 2013 4:33 pm UTC

skeptical scientist wrote:
You, sir, name? wrote:I wish I had VS problems.

Yesterday, I tried to run a JUit test in eclipse. [...]

It could be worse. You could be using XCode.

Is it that bad? I never really tried it.
"There are only two hard problems in computer science: cache coherence, naming things, and off-by-one errors." (Phil Karlton and Leon Bambrick)

coding and xkcd combined

(Julian/Julian's)

User avatar
Xenomortis
Not actually a special flower.
Posts: 1456
Joined: Thu Oct 11, 2012 8:47 am UTC

Re: Coding: Fleeting Thoughts

Postby Xenomortis » Tue Sep 10, 2013 5:37 pm UTC

You, sir, name? wrote:My previous employer had no testing at all. Each version got to run in a staging environment (where at best it got some monkey testing) before being deployed onto production. If shit hit the fan, you fixed it *in production.

Not a small UK based web company by any chance?

I did just have this conversation a few hours ago with one the senior developer:

"Me: So I think this change will fix this issue*, but since I don't really know precisely what data we'll get for a given customer's state nor how it will interact for this other set of data** (which cannot easily be tested), I'm not at all confident"
"Other: Well, it appears to work for your test customer, so we could just put it live and see if we get any complaints"
"Me: I don't really like the sound of that"

*displaying and interpreting some customer data given to us by a third party
**from another third party

Good thing we didn't leave it there; only later, and after examining some real data, did we get some instruction as to what should actually be displayed (albeit verbally).

But this is all minor. There have been other, far more concerning, things I've seen and overheard; although I'm not comfortable going into any details.

I wonder if this is common. I can't exactly tell potential employers apart from job ads and TDWTF doesn't exactly fill me with confidence.
Image

User avatar
skeptical scientist
closed-minded spiritualist
Posts: 6142
Joined: Tue Nov 28, 2006 6:09 am UTC
Location: San Francisco

Re: Coding: Fleeting Thoughts

Postby skeptical scientist » Tue Sep 10, 2013 10:22 pm UTC

Jplus wrote:Is it that bad? I never really tried it.

Things I hate about XCode:
  • It randomly hangs while I'm not doing anything but typing text and spins a pinwheel for a while (when it doesn't just crash).
  • It stores a ton of configuration data under the hood where it is inaccessible except through a complicated GUI so you have to search through the GUI to figure out why things are breaking even though the code you wrote is obviously correct.
  • It maintains its own directory structure for projects which can have nothing to do with either the actual filesystem directory structure or which files belong to which targets. Why???
  • It lets me accidentally edit read-only files, so it's constantly fighting with perforce.
  • If I close windows in the wrong order, next time I start xcode I have to tell it which interface panels I want to see all over again.
I'm looking forward to the day when the SNES emulator on my computer works by emulating the elementary particles in an actual, physical box with Nintendo stamped on the side.

"With math, all things are possible." —Rebecca Watson

User avatar
You, sir, name?
Posts: 6983
Joined: Sun Apr 22, 2007 10:07 am UTC
Location: Chako Paul City
Contact:

Re: Coding: Fleeting Thoughts

Postby You, sir, name? » Wed Sep 11, 2013 8:25 pm UTC

Xenomortis wrote:
You, sir, name? wrote:My previous employer had no testing at all. Each version got to run in a staging environment (where at best it got some monkey testing) before being deployed onto production. If shit hit the fan, you fixed it *in production.

Not a small UK based web company by any chance?

I did just have this conversation a few hours ago with one the senior developer:

Spoiler:
"Me: So I think this change will fix this issue*, but since I don't really know precisely what data we'll get for a given customer's state nor how it will interact for this other set of data** (which cannot easily be tested), I'm not at all confident"
"Other: Well, it appears to work for your test customer, so we could just put it live and see if we get any complaints"
"Me: I don't really like the sound of that"

*displaying and interpreting some customer data given to us by a third party
**from another third party

Good thing we didn't leave it there; only later, and after examining some real data, did we get some instruction as to what should actually be displayed (albeit verbally).


But this is all minor. There have been other, far more concerning, things I've seen and overheard; although I'm not comfortable going into any details.

I wonder if this is common. I can't exactly tell potential employers apart from job ads and TDWTF doesn't exactly fill me with confidence.


Nope, this was a large multinational entity in finance. Stupid amounts of money hinged upon this software working properly.

I think in general this sort of practice is more common in in-house software development for businesses that don't sell code. Pure software development shops seem in my experience more likely to have some concept of good practices, if nothing else because not having so may result in expensive lawsuits.
I edit my posts a lot and sometimes the words wrong order words appear in sentences get messed up.

User avatar
Diadem
Posts: 5654
Joined: Wed Jun 11, 2008 11:03 am UTC
Location: The Netherlands

Re: Coding: Fleeting Thoughts

Postby Diadem » Wed Sep 11, 2013 8:36 pm UTC

So here's a question for y'all.

I've got c++ code where the coding style is all over the place. For example identing is done willy-nilly with spaces or tabs, sometimes jumping to three different levels within the same code block. It's a mess. Cleaning this up by hand will take a long time, it's several hundred thousand lines of code, all in all.

I know there are automated tools for this job. But how good / reliable are they? Is there a chance of introducing bugs with them? Does the formatting sometimes come out weird? And which ones are best? Anybody got experience with such a tool?
It's one of those irregular verbs, isn't it? I have an independent mind, you are an eccentric, he is round the twist
- Bernard Woolley in Yes, Prime Minister

User avatar
skeptical scientist
closed-minded spiritualist
Posts: 6142
Joined: Tue Nov 28, 2006 6:09 am UTC
Location: San Francisco

Re: Coding: Fleeting Thoughts

Postby skeptical scientist » Wed Sep 11, 2013 11:13 pm UTC

I've used clang-format (http://clang.llvm.org/docs/ClangFormat.html) with success.
I'm looking forward to the day when the SNES emulator on my computer works by emulating the elementary particles in an actual, physical box with Nintendo stamped on the side.

"With math, all things are possible." —Rebecca Watson

User avatar
Xeio
Friends, Faidites, Countrymen
Posts: 5101
Joined: Wed Jul 25, 2007 11:12 am UTC
Location: C:\Users\Xeio\
Contact:

Re: Coding: Fleeting Thoughts

Postby Xeio » Thu Sep 12, 2013 2:48 am UTC

Turns out if you don't prune your quad tree, it runs several orders of magnitude slower.

Who'd have thunk?

User avatar
Xenomortis
Not actually a special flower.
Posts: 1456
Joined: Thu Oct 11, 2012 8:47 am UTC

Re: Coding: Fleeting Thoughts

Postby Xenomortis » Thu Sep 12, 2013 9:59 am UTC

I have Visual Studio set to replace Tabs with Spaces. I've made some small changes to a few code files, each containing several thousand lines.
Now, our code-base isn't exactly consistent with Tabs and Spaces, although the sizes don't usually differ so it's not a big deal. A lot of code in these files was written using tabs, not spaces (and some was written with spaces).
Somehow, whilst making my changes, VS replaced a lot of tabs with spaces. My fingerprints are now over random blocks of code that I made no change to.
Oh, and the line-numbers on a lot of them have been changed by one.

On the bright side, nobody else here appears to know about the various features of svn, or source control in general (like Blame, Diff), so I think I'm safe.
Image

User avatar
Pesto
Posts: 737
Joined: Wed Sep 05, 2007 5:33 pm UTC
Location: Berkeley, CA

Re: Coding: Fleeting Thoughts

Postby Pesto » Thu Sep 12, 2013 5:10 pm UTC

I can find no redeeming qualities in XML.

User avatar
You, sir, name?
Posts: 6983
Joined: Sun Apr 22, 2007 10:07 am UTC
Location: Chako Paul City
Contact:

Re: Coding: Fleeting Thoughts

Postby You, sir, name? » Thu Sep 12, 2013 5:58 pm UTC

I think it may be argued that XML is alive. Some criteria from wikipedia:

Metabolism: The XML monster eats terabytes of other data every day. Only XML emerges on the other end.
Growth: Growing is pretty much what XML does.
Organization: The DTD.
Response to stimuli: Some dialects, like XHTML, can be highly responsive to stimuli.
Reproduction: XML creates more XML. If you're having XML trouble, more XML is almost certainly the solution. You frequently use XML to create XML from XML.

It's only homeostasis and adaptation that are left.
I edit my posts a lot and sometimes the words wrong order words appear in sentences get messed up.

User avatar
Link
Posts: 1419
Joined: Sat Mar 07, 2009 11:33 am UTC
Location: ᘝᓄᘈᖉᐣ
Contact:

Re: Coding: Fleeting Thoughts

Postby Link » Thu Sep 12, 2013 6:30 pm UTC

You, sir, name? wrote:It's only homeostasis and adaptation that are left.
Adaptation is covered, IMO. XML can take many different forms suitable to whichever environment it happens to encounter. In fact, I'd call it a polyextremophile: it's hard to think of an environment where it can't thrive. It's about as unkillable as tardigrades, and worse yet, as invasive as the cane toad.

User avatar
You, sir, name?
Posts: 6983
Joined: Sun Apr 22, 2007 10:07 am UTC
Location: Chako Paul City
Contact:

Re: Coding: Fleeting Thoughts

Postby You, sir, name? » Thu Sep 12, 2013 7:55 pm UTC

It's been a weird-ass year. I've gotten a series of very large development tasks done, mostly on my own. They were chained by sentiments like "Wow! You did an awesome job building X in a short time, why don't you build Y, also in a short time?", and because of this, I wrote a rather large chunk of the code base and earned a sort of de facto title of senior developer (I'm one of the youngest developers on the payroll). I'm now the guy who knows everything about X, Y, Z and W (and other things I supposedly know but can only make educated guesses about). I'm the guy who probably knows stuff about stuff.

Sounds good, right? Well. I thought so too. But lately, in practice it means I some days spend more time tutoring people (including supposed "tech leads") on relatively basic aspects of our code base, and in meetings with BAs and clients; than I do actually coding.

Days like these I dread I teeter on the verge of some managerial role, my job description sucked through the Enigma of Amigara Fault, to emerge distorted into a mindless Excel jockey.

I just want to write cool code :-(
I edit my posts a lot and sometimes the words wrong order words appear in sentences get messed up.

User avatar
Pesto
Posts: 737
Joined: Wed Sep 05, 2007 5:33 pm UTC
Location: Berkeley, CA

Re: Coding: Fleeting Thoughts

Postby Pesto » Thu Sep 12, 2013 8:06 pm UTC

You, sir, name? wrote:I think it may be argued that XML is alive.

There is no doubt to that fact. I'm unconvinced that this is a redeeming quality.

User avatar
You, sir, name?
Posts: 6983
Joined: Sun Apr 22, 2007 10:07 am UTC
Location: Chako Paul City
Contact:

Re: Coding: Fleeting Thoughts

Postby You, sir, name? » Thu Sep 12, 2013 8:19 pm UTC

I edit my posts a lot and sometimes the words wrong order words appear in sentences get messed up.

User avatar
Xeio
Friends, Faidites, Countrymen
Posts: 5101
Joined: Wed Jul 25, 2007 11:12 am UTC
Location: C:\Users\Xeio\
Contact:

Re: Coding: Fleeting Thoughts

Postby Xeio » Thu Sep 12, 2013 9:20 pm UTC

Pesto wrote:I can find no redeeming qualities in XML.
Uh... it's human readable?

Not that there aren't other, better, human readable and editable formats...

EvanED
Posts: 4331
Joined: Mon Aug 07, 2006 6:28 am UTC
Location: Madison, WI
Contact:

Re: Coding: Fleeting Thoughts

Postby EvanED » Thu Sep 12, 2013 9:27 pm UTC

Xeio wrote:Uh... it's barely human readable?
FTFM :-)

bittyx
Posts: 194
Joined: Tue Sep 25, 2007 9:10 pm UTC
Location: Belgrade, Serbia

Re: Coding: Fleeting Thoughts

Postby bittyx » Thu Sep 12, 2013 11:07 pm UTC

You, sir, name? wrote:http://www.youtube.com/watch?v=TdUsyXQ8Wrs

That's the most accurate decription of XML I've come across so far.


Return to “Coding”

Who is online

Users browsing this forum: No registered users and 5 guests